3 months agoFix Cerberus error master production
Sybren A. Stüvel [Thu, 19 Mar 2020 16:04:13 +0000 (17:04 +0100)]
Fix Cerberus error

Same error (and same fix) as was fixed in Pillar 7dc0cadc.

3 months agoAdded `delete_orphan_task_logs` CLI command
Sybren A. Stüvel [Thu, 19 Mar 2020 15:07:33 +0000 (16:07 +0100)]
Added `delete_orphan_task_logs` CLI command

It fetches the ID and the Task ID for *every* task log entry, and performs
a 'does this task exist' check in Python. The logic is in Python because I
couldn't find a MongoDB query that allows "foreign key" lookups and delete
documents where this cannot find the referenced document.

3 months agoCleanup: use named constants instead of literals in test
Sybren A. Stüvel [Thu, 19 Mar 2020 13:52:08 +0000 (14:52 +0100)]
Cleanup: use named constants instead of literals in test

No functional changes.

3 months agoFix job archival not removing old task logs
Sybren A. Stüvel [Thu, 19 Mar 2020 13:13:40 +0000 (14:13 +0100)]
Fix job archival not removing old task logs

11 months agoDocs: Update architecture diagram
Francesco Siddi [Thu, 25 Jul 2019 08:49:12 +0000 (10:49 +0200)]
Docs: Update architecture diagram

Extend the Remote Farm section specifying an additional cloud vendor.

13 months agoReplaced Gravatar with self-hosted avatars
Sybren A. Stüvel [Fri, 31 May 2019 11:54:21 +0000 (13:54 +0200)]
Replaced Gravatar with self-hosted avatars

Requires Pillar 47474ac936ffb1d179161c8a3cac5d20e6005659

13 months agoUpgraded Gulp 3.9.1 → 4.0.2
Sybren A. Stüvel [Wed, 29 May 2019 14:46:53 +0000 (16:46 +0200)]
Upgraded Gulp 3.9.1 → 4.0.2

This was a result of running `npm audit fix --force`.

13 months agoRemoved livereload from NPM/Gulp + relocked NPM dependencies
Sybren A. Stüvel [Wed, 29 May 2019 14:46:01 +0000 (16:46 +0200)]
Removed livereload from NPM/Gulp + relocked NPM dependencies

Use your editor's support, it's better ;-)

13 months agoCompatibility with Cerberus 1.3
Sybren A. Stüvel [Wed, 29 May 2019 14:05:38 +0000 (16:05 +0200)]
Compatibility with Cerberus 1.3

See Pillar 8b42e888175239fdac93b408e627bea0533f6473

13 months agoMongoDB: cursor.count() deprecated in favour of coll.count_documents()
Sybren A. Stüvel [Wed, 29 May 2019 14:05:25 +0000 (16:05 +0200)]
MongoDB: cursor.count() deprecated in favour of coll.count_documents()

Note that `ManagerManager.owned_managers` changed return type because of
this change.

13 months agoReplaced unittest.assertEquals → assertEqual
Sybren A. Stüvel [Wed, 29 May 2019 14:04:14 +0000 (16:04 +0200)]
Replaced unittest.assertEquals → assertEqual

13 months agoRemoved and gitignored poetry.lock
Sybren A. Stüvel [Thu, 23 May 2019 14:23:45 +0000 (16:23 +0200)]
Removed and gitignored poetry.lock

The poetry.lock files are only relevant for repeatable deployments,
and the one in this project isn't used for that (only the Blender
Cloud project file is used, and that's still there).

13 months agoRe-locked dependencies after Pillar dep upgrade
Sybren A. Stüvel [Tue, 14 May 2019 12:40:16 +0000 (14:40 +0200)]
Re-locked dependencies after Pillar dep upgrade

14 months agoUpdated update_version.sh to use Poetry
Sybren A. Stüvel [Fri, 10 May 2019 13:18:26 +0000 (15:18 +0200)]
Updated update_version.sh to use Poetry

14 months agoREADME+CHANGELOG: documented use of Poetry
Sybren A. Stüvel [Fri, 10 May 2019 13:07:38 +0000 (15:07 +0200)]
README+CHANGELOG: documented use of Poetry

14 months agoDev-deps in poetry
Sybren A. Stüvel [Fri, 26 Apr 2019 10:16:27 +0000 (12:16 +0200)]
Dev-deps in poetry

14 months agoUsing Poetry to handle packaging
Sybren A. Stüvel [Fri, 26 Apr 2019 09:46:11 +0000 (11:46 +0200)]
Using Poetry to handle packaging

14 months agoNo longer using deprecated db.collection_names() function
Sybren A. Stüvel [Fri, 26 Apr 2019 09:45:50 +0000 (11:45 +0200)]
No longer using deprecated db.collection_names() function

14 months agoMade some comments & code reflect the current reality
Sybren A. Stüvel [Thu, 25 Apr 2019 11:59:32 +0000 (13:59 +0200)]
Made some comments & code reflect the current reality

14 months agoAlso clear out failed_by_workers when using 'requeue failed tasks' on job
Sybren A. Stüvel [Thu, 25 Apr 2019 09:14:59 +0000 (11:14 +0200)]
Also clear out failed_by_workers when using 'requeue failed tasks' on job

14 months agoRe-queueing a task now erases its `failed_by_workers` list
Sybren A. Stüvel [Thu, 25 Apr 2019 08:50:13 +0000 (10:50 +0200)]
Re-queueing a task now erases its `failed_by_workers` list

The 'failed_by_workers' list is also erased when the entire job is

Erasing the list ensures that workers that previously failed the task
can try again. This is essential when the task was failing due to
external conditions, for example failure due to a bug in Flamenco Worker
that's since been resolved.

14 months agoGracefully handle missing 'flamenco_last_project' in session
Sybren A. Stüvel [Thu, 18 Apr 2019 15:29:00 +0000 (17:29 +0200)]
Gracefully handle missing 'flamenco_last_project' in session

14 months agoPrevent Manager settings downgrade when already old enough
Sybren A. Stüvel [Thu, 18 Apr 2019 15:15:12 +0000 (17:15 +0200)]
Prevent Manager settings downgrade when already old enough

The code tries to convert version 2 to version 1 settings (for older
Blender Cloud add-ons that require version 1) even when the Manager was
using version 1 settings.

14 months agoDocument new-style variables
Sybren A. Stüvel [Thu, 18 Apr 2019 11:53:08 +0000 (13:53 +0200)]
Document new-style variables

14 months agoSupport version 2 of Flamenco Manager variables
Sybren A. Stüvel [Thu, 18 Apr 2019 08:49:54 +0000 (10:49 +0200)]
Support version 2 of Flamenco Manager variables

A restructuring of the Manager variables was necessary to allow
different values for users and workers. Until now both had the same
paths, which is fine when both are located in the same studio. However,
when deploying workers on a cloud infrastructure (Azure, AWS, etc.) this
becomes cumbersome.

When a Manager does not declare a `settings_version`, version 1 is
assumed. This commit introduces version 2.

The naming 'path replacement variable' was confusing people. Now each
variable has a mandatory property 'direction' that has one of the
following values:

- `"one"`: unidirectional variable. `${VARIABLENAME}` is replaced by the
  variable value for the appropriate platform. This used to be the
- `"two"`: bidirection variable. As described above, but also the value
  is replaced by `${VARIABLENAME}` by the Blender Cloud add-on when
  sending a job to Flamenco Server. This used to be the 'path
  replacement variable'.

Each variable now has a setting 'audience', which can be either
'workers' or 'users' to be applied only to those, or 'all' when the
value is the same for both.

15 months agoClean up: Whitespace
Tobias Johansson [Thu, 4 Apr 2019 09:34:13 +0000 (11:34 +0200)]
Clean up: Whitespace

15 months agoUse kebab-case for vue names
Tobias Johansson [Thu, 4 Apr 2019 09:33:42 +0000 (11:33 +0200)]
Use kebab-case for vue names


15 months agoSilence warning about changing prop value
Tobias Johansson [Thu, 4 Apr 2019 08:18:24 +0000 (10:18 +0200)]
Silence warning about changing prop value

15 months agoWhitespace cleanup
Tobias Johansson [Wed, 3 Apr 2019 15:51:28 +0000 (17:51 +0200)]
Whitespace cleanup

15 months agoBetter initial component values
Tobias Johansson [Wed, 3 Apr 2019 15:50:14 +0000 (17:50 +0200)]
Better initial component values

15 months agoDocumentation of es6 transcompile and packaging
Tobias Johansson [Fri, 29 Mar 2019 09:44:03 +0000 (10:44 +0100)]
Documentation of es6 transcompile and packaging

15 months agoAdd Created and Updated column
Tobias Johansson [Thu, 28 Mar 2019 11:48:15 +0000 (12:48 +0100)]
Add Created and Updated column

15 months agoStore filter/column settings in localStorage
Tobias Johansson [Thu, 28 Mar 2019 09:29:13 +0000 (10:29 +0100)]
Store filter/column settings in localStorage

The filter and column settings in tables are stored per project and
context in the browsers localStorage. This makes the table keep the
settings even if the browser is refreshed or restarted.

The table emits a "componentStateChanged" event containing the tables
current state (filter/column settings) which then is saved by the top
level component.

15 months agoUpdate gulp-sass to 4.0.2 in package.json
Francesco Siddi [Wed, 27 Mar 2019 13:53:41 +0000 (14:53 +0100)]
Update gulp-sass to 4.0.2 in package.json

Version 4.0.1 was failing to build libsass on macOS.

15 months agoNPM decided to add a LOT of new dependencies to package-lock.json
Sybren A. Stüvel [Tue, 26 Mar 2019 10:26:58 +0000 (11:26 +0100)]
NPM decided to add a LOT of new dependencies to package-lock.json

15 months agoProper indentation in package.json
Sybren A. Stüvel [Tue, 26 Mar 2019 10:26:18 +0000 (11:26 +0100)]
Proper indentation in package.json

15 months agoUsing Toastr instead of `statusBarSet()`
Sybren A. Stüvel [Tue, 26 Mar 2019 10:26:08 +0000 (11:26 +0100)]
Using Toastr instead of `statusBarSet()`

The former Just Works™, whereas the latter only works if there is no
task opened in the left-hand part of the page.

15 months agoAdded Shaman documentation
Sybren A. Stüvel [Mon, 25 Mar 2019 16:42:03 +0000 (17:42 +0100)]
Added Shaman documentation

15 months agoUpdated docs dependencies
Sybren A. Stüvel [Mon, 25 Mar 2019 16:30:16 +0000 (17:30 +0100)]
Updated docs dependencies

15 months agoBumped version to 2.3.dev0
Sybren A. Stüvel [Mon, 25 Mar 2019 15:32:44 +0000 (16:32 +0100)]
Bumped version to 2.3.dev0

15 months agoBumped version to 2.2 v2.2
Sybren A. Stüvel [Mon, 25 Mar 2019 15:21:21 +0000 (16:21 +0100)]
Bumped version to 2.2

15 months agoMark version 2.2 as released
Sybren A. Stüvel [Mon, 25 Mar 2019 15:21:14 +0000 (16:21 +0100)]
Mark version 2.2 as released

15 months agoMake sure sort buttons is always clickable
Tobias Johansson [Fri, 22 Mar 2019 13:10:18 +0000 (14:10 +0100)]
Make sure sort buttons is always clickable

Hide part overflow of column label if there is not enough room

15 months agoAdd missing tooltips in table
Tobias Johansson [Fri, 22 Mar 2019 13:07:28 +0000 (14:07 +0100)]
Add missing tooltips in table

15 months agoUse monospace font for flamenco table
Tobias Johansson [Thu, 21 Mar 2019 09:53:18 +0000 (10:53 +0100)]
Use monospace font for flamenco table

15 months agoFlamenco jobs table filter/sortable
Tobias Johansson [Thu, 21 Mar 2019 08:59:36 +0000 (09:59 +0100)]
Flamenco jobs table filter/sortable

Using vue pillar table for flamenco (archived-)jobs.
Multi-select in table is disabled since at the moment multi-editing
is not implemented for flamenco jobs.

15 months agoAdded es6 trans compiling
Tobias Johansson [Thu, 21 Mar 2019 08:54:37 +0000 (09:54 +0100)]
Added es6 trans compiling

15 months agoAllow jwt/get-token endpoint also when authing using Basic auth
Sybren A. Stüvel [Tue, 19 Mar 2019 11:38:59 +0000 (12:38 +0100)]
Allow jwt/get-token endpoint also when authing using Basic auth

The Pillar API uses basic auth, not Bearer tokens (even though tokens are

15 months agoUse Pillar's new `cors.allow()` decorator
Sybren A. Stüvel [Tue, 19 Mar 2019 10:19:22 +0000 (11:19 +0100)]
Use Pillar's new `cors.allow()` decorator

See Pillar a104117618da0a4974fca622aab5864e1f234b0b

15 months agoFix: generate JWT expiry timestamps as UTC
Sybren A. Stüvel [Tue, 19 Mar 2019 10:19:06 +0000 (11:19 +0100)]
Fix: generate JWT expiry timestamps as UTC

Apparently `datetime.datetime.utcnow().timestamp()` does NOT return a UTC
timestamp. `utcnow()` returns a timezone-unaware `datetime`, which
`timestamp()` then interprets as local time and incorrectly converts again
to a UTC timestamp.

15 months agoUnit tests for JWT token generation
Sybren A. Stüvel [Mon, 18 Mar 2019 15:06:36 +0000 (16:06 +0100)]
Unit tests for JWT token generation

15 months agoJWT token generation: only send CORS headers in reply when requested
Sybren A. Stüvel [Mon, 18 Mar 2019 15:06:26 +0000 (16:06 +0100)]
JWT token generation: only send CORS headers in reply when requested

We now only send CORS header in the reply when the request has an Origin
header (indicating it's a cross-site request).

15 months agoJWT validation: don't trust links that are valid for too long
Sybren A. Stüvel [Mon, 18 Mar 2019 15:05:51 +0000 (16:05 +0100)]
JWT validation: don't trust links that are valid for too long

Flamenco Manager makes links valid for less than a minute. If we get a link
that is valid for longer than 15 minutes, it's been doctored and we should
not trust it.

15 months agoAllow CORS requests from a Manager to the JWT token generator URL
Sybren A. Stüvel [Mon, 18 Mar 2019 13:37:25 +0000 (14:37 +0100)]
Allow CORS requests from a Manager to the JWT token generator URL

These requests are authenticated by the user session. In other words, the
user should visit Flamenco Server in their webbrowser and log in, before
a token is generated. While the session is valid any number of tokens can
be generated.

To avoid CSRF attacks, a request that uses the session to authenticate the
user MUST include an expiry timestamp

15 months agoJWT public keys: format Last-Modified header as 'HTTP date'
Sybren A. Stüvel [Wed, 13 Mar 2019 15:18:12 +0000 (16:18 +0100)]
JWT public keys: format Last-Modified header as 'HTTP date'

15 months agoSolved a bunch of MyPy errors
Sybren A. Stüvel [Wed, 13 Mar 2019 09:51:41 +0000 (10:51 +0100)]
Solved a bunch of MyPy errors

15 months agoAdded JWT key store
Sybren A. Stüvel [Thu, 7 Mar 2019 16:15:01 +0000 (17:15 +0100)]
Added JWT key store

This key store contains a private key used to sign JWT tokens, and a set
of public keys. One of the public keys should match the private key.
Support for other public keys is there to allow a rolling changeover of

- Create a new private+public key pair and place aside.
- Append the new public key to the key store.
- Wait until all Flamenco Managers have had a chance to download the new
  set of public keys.
- Replace the private key with the newly generated one.
- Remove the old public key from the key store.

15 months agoReturn empty string instead of None to indicate 'no status change'
Sybren A. Stüvel [Wed, 13 Mar 2019 09:52:11 +0000 (10:52 +0100)]
Return empty string instead of None to indicate 'no status change'

15 months agoRemoved unused import
Sybren A. Stüvel [Wed, 13 Mar 2019 08:56:21 +0000 (09:56 +0100)]
Removed unused import

15 months agoUpdated CHANGELOG
Sybren A. Stüvel [Tue, 12 Mar 2019 14:19:47 +0000 (15:19 +0100)]

15 months agoBlender Render Progressive: don't generate video when rendering one frame
Sybren A. Stüvel [Tue, 12 Mar 2019 14:16:51 +0000 (15:16 +0100)]
Blender Render Progressive: don't generate video when rendering one frame

15 months agoBlender Render: don't create video when there is only one frame to render
Sybren A. Stüvel [Tue, 12 Mar 2019 13:55:24 +0000 (14:55 +0100)]
Blender Render: don't create video when there is only one frame to render

15 months agoBlender Render: do not create preview video when rendering EXR files
Sybren A. Stüvel [Tue, 12 Mar 2019 13:43:47 +0000 (14:43 +0100)]
Blender Render: do not create preview video when rendering EXR files

The EXR files can be at a higher bit-depth than FFmpeg can handle (by
default). Furthermore, the colours will likely be wrong due to EXR having
linear colours, and FFmpeg not having the correct colour space conversion.

15 months agoUse DNA enum values for render format in blender render jobs
Sybren A. Stüvel [Tue, 12 Mar 2019 13:42:55 +0000 (14:42 +0100)]
Use DNA enum values for render format in blender render jobs

DNA uses `OPEN_EXR` whereas the `--render-format` CLI parameter uses `EXR`.
The job settings should be using the DNA setting, whereas the conversion
to a CLI parameter should be done by the Worker.

See https://developer.blender.org/D4502

15 months agoSplit test code from test_job_compilers into test_job_compiler_blender_render
Sybren A. Stüvel [Tue, 12 Mar 2019 12:31:42 +0000 (13:31 +0100)]
Split test code from test_job_compilers into test_job_compiler_blender_render

16 months agoBlender Video Chunks: take output extension into account
Sybren A. Stüvel [Wed, 6 Mar 2019 12:28:04 +0000 (13:28 +0100)]
Blender Video Chunks: take output extension into account

We now allow the client to set the final output extension. This is limited
to what FFmpeg understands, though, so it should be used with care. The
Blender Cloud add-on checks for supported extensions already.

16 months agoBlender-video-chunks job compiler: output to JPEG instead of PNG
Sybren A. Stüvel [Fri, 1 Mar 2019 14:43:39 +0000 (15:43 +0100)]
Blender-video-chunks job compiler: output to JPEG instead of PNG

16 months agoAutomatically delete jobs stuck in `waiting-for-files` status
Sybren A. Stüvel [Thu, 28 Feb 2019 13:14:07 +0000 (14:14 +0100)]
Automatically delete jobs stuck in `waiting-for-files` status

These jobs are waiting for an external PATCH call to initiate job
compilation, queueing, and execution. If this PATCH call doesn't come,
the job is stuck in this status. After a certain time of waiting, this
function will automatically delete those jobs.

Be sure to add a schedule to the Celery Beat like this:

    'remove_waiting_for_files': {
        'task': 'flamenco.celery.job_cleanup.remove_waiting_for_files',
        'schedule': 3600,  # every N seconds

16 months agoAllow jobs to be created with 'waiting-for-files' status.
Sybren A. Stüvel [Tue, 26 Feb 2019 17:03:18 +0000 (18:03 +0100)]
Allow jobs to be created with 'waiting-for-files' status.

This prohibits task compilation and job queueing. It requires a PATCH
with `{'op': 'construct'}` to send it to `under-construction` status and
initiate task compilation.

This has no impact on the `start_paused` setting; it behaves as before.

16 months agobson.ObjectId → ObjectId
Sybren A. Stüvel [Tue, 26 Feb 2019 16:42:49 +0000 (17:42 +0100)]
bson.ObjectId → ObjectId

16 months agoFixed sidebar links when there is no 'last project' in the session
Sybren A. Stüvel [Tue, 26 Feb 2019 10:12:33 +0000 (11:12 +0100)]
Fixed sidebar links when there is no 'last project' in the session

16 months agoUI: Remove the list header.
Pablo Vazquez [Fri, 22 Feb 2019 01:52:19 +0000 (02:52 +0100)]
UI: Remove the list header.

Comment out the header until stats and filtering is implemented.

It currently contains 'infinite' stats, and a link to the project's
homepage which sits right above it since the latest navigation update.

16 months agoUI: Don't use white as fallback for unknown status-indicator
Pablo Vazquez [Fri, 22 Feb 2019 01:35:10 +0000 (02:35 +0100)]
UI: Don't use white as fallback for unknown status-indicator

Some new status would be invisible in the list, since they don't have a
specific color set yet and the background of the list is white.

16 months agoUI: Margin on buttons
Pablo Vazquez [Fri, 22 Feb 2019 01:33:51 +0000 (02:33 +0100)]
UI: Margin on buttons

16 months agoUI: Align Cancel/View Log buttons to the left.
Pablo Vazquez [Fri, 22 Feb 2019 01:33:18 +0000 (02:33 +0100)]
UI: Align Cancel/View Log buttons to the left.

Also don't use 'success' buttons for actions like request logs,
save the green color for creation/re-queueing type of tasks (aligned to the right).

16 months agoUI: Align Cancel/Archive/RNA buttons to the left.
Pablo Vazquez [Fri, 22 Feb 2019 01:32:05 +0000 (02:32 +0100)]
UI: Align Cancel/Archive/RNA buttons to the left.

Also don't use 'success' buttons for actions like Archive or Edit RNA,
save the green color for creation/re-queueing type of tasks.

16 months agoUI: Task list, align navigation to the right
Pablo Vazquez [Fri, 22 Feb 2019 01:29:38 +0000 (02:29 +0100)]
UI: Task list, align navigation to the right

16 months agoInclude task timing metrics
Sybren A. Stüvel [Thu, 21 Feb 2019 16:39:30 +0000 (17:39 +0100)]
Include task timing metrics

This includes receiving them from the Manager and presenting them on the
task view.

16 months agoTask log requests: always de-queue request when Manager sends us a log
Sybren A. Stüvel [Thu, 21 Feb 2019 09:59:18 +0000 (10:59 +0100)]
Task log requests: always de-queue request when Manager sends us a log

Even when the task doesn't exist any more, and we can't process the log,
the request to send the log should be de-queued to prevent infinite loops.

16 months agoView task template: added space between parent ObjectIDs
Sybren A. Stüvel [Wed, 20 Feb 2019 09:17:11 +0000 (10:17 +0100)]
View task template: added space between parent ObjectIDs

This makes it possible for the browser to wrap long lines when a task has
many parents.

16 months agoReplaced 'status'-legend popup with mouse-over that actually explains stuff
Sybren A. Stüvel [Tue, 19 Feb 2019 16:02:42 +0000 (17:02 +0100)]
Replaced 'status'-legend popup with mouse-over that actually explains stuff

Fix T61664

16 months agoRemoved `openModalUrl` jQuery function
Sybren A. Stüvel [Tue, 19 Feb 2019 15:44:40 +0000 (16:44 +0100)]
Removed `openModalUrl` jQuery function

It is broken because `$(...).modal()` doesn't exist, and there is no
comment about which library should provide it. Furthermore, it's only used
to load endpoints that no longer seem to work either, so I removed those
as well.

16 months agoAdded 'failed_by_workers' property to tasks
Sybren A. Stüvel [Tue, 19 Feb 2019 15:22:16 +0000 (16:22 +0100)]
Added 'failed_by_workers' property to tasks

This is set by the Manager to indicate which workers have run this task and

16 months agoBetter readability of progressive render tasks
Sybren A. Stüvel [Thu, 14 Feb 2019 14:49:12 +0000 (15:49 +0100)]
Better readability of progressive render tasks

`render-smplA-B-frmX-Y` → `render-s_A-B-f_X-Y`
Same with merge tasks.

16 months agoIntroduced task status 'soft-failed'
Sybren A. Stüvel [Thu, 14 Feb 2019 10:29:11 +0000 (11:29 +0100)]
Introduced task status 'soft-failed'

This status indicates that there was a failure, but that the task will be
re-tried by another worker. As such, it should be handled fairly similarly
to 'claimed-by-manager'. Soft-failed tasks shouldn't affect the job status.

16 months agoAdded missing CSS colours for fail-requested status
Sybren A. Stüvel [Thu, 14 Feb 2019 10:27:11 +0000 (11:27 +0100)]
Added missing CSS colours for fail-requested status

16 months agoJob archival: don't crash on tasks without log
Sybren A. Stüvel [Thu, 14 Feb 2019 09:26:47 +0000 (10:26 +0100)]
Job archival: don't crash on tasks without log

16 months agoRunnability check: ensure tasks are counted only once
Sybren A. Stüvel [Wed, 13 Feb 2019 14:48:23 +0000 (15:48 +0100)]
Runnability check: ensure tasks are counted only once

Before this fix, each task was counted for each of its failed parents (so
a task with 3 failed parents would be counted 3 times).

16 months ago'fail-requested' job should stay 'fail-requested' even when tasks are active
Sybren A. Stüvel [Wed, 13 Feb 2019 14:35:43 +0000 (15:35 +0100)]
'fail-requested' job should stay 'fail-requested' even when tasks are active

Before, when a task update with status='active' would come in for a job
with status='fail-requested', the job status would be set to 'active'.
This is fixed now.

16 months agoVendoring Bowser rather than copying it from node_modules
Sybren A. Stüvel [Wed, 13 Feb 2019 13:55:25 +0000 (14:55 +0100)]
Vendoring Bowser rather than copying it from node_modules

It seems to work fine, except during staging for building the docker image;
then it silently fails to copy the bowser.js file into Flamenco's `static`

16 months agoJob view: show user and creation timestamp more explicitly
Sybren A. Stüvel [Wed, 13 Feb 2019 13:28:55 +0000 (14:28 +0100)]
Job view: show user and creation timestamp more explicitly

Previously only the username was shown (no full name), and the creation
timestamp was hidden behind the updated timestamp.

16 months agoAllow users to copy-paste job settings
Sybren A. Stüvel [Wed, 13 Feb 2019 13:22:40 +0000 (14:22 +0100)]
Allow users to copy-paste job settings

Before copying, variables in the job setting value are replaced based on
the user's own platform. Furthermore, specific settings can be processed
before copying; for example, the `render_output` setting gets its `######`
removed and a directory popped so that you end up at the directory
containing the intermediate directory and/or the final directory.

16 months agoRegression Fix: Choose flamenco manager broke while unifying ui.
Tobias Johansson [Wed, 13 Feb 2019 09:29:17 +0000 (10:29 +0100)]
Regression Fix: Choose flamenco manager broke while unifying ui.

Found using sentry.

16 months agoRegression Fix: Overflow in column scroll. Updated ugly hack.
Tobias Johansson [Wed, 13 Feb 2019 08:19:28 +0000 (09:19 +0100)]
Regression Fix: Overflow in column scroll. Updated ugly hack.

16 months agoSet job's 'status_reason' when recreating job
Sybren A. Stüvel [Tue, 12 Feb 2019 16:59:11 +0000 (17:59 +0100)]
Set job's 'status_reason' when recreating job

When recreating a job, its 'status_reason' field should be set,
otherwise it would keep it from the old run (and be meaningless at

16 months agoFix T50572 Detect unrunnable active jobs
Sybren A. Stüvel [Tue, 12 Feb 2019 16:58:57 +0000 (17:58 +0100)]
Fix T50572 Detect unrunnable active jobs

Active Flamenco jobs are periodically checked (via Celery) for non-runnable
tasks (e.g. queued tasks with a failed/cancelled parent). When such a task
is found, the entire job is failed (because it won't be able to complete

Add something like this to the CELERY_BEAT_SCHEDULE setting:

    'job-runnability-check': {
        'task': 'flamenco.celery.job_runnability_check.schedule_checks',
        'schedule': 600,  # every N seconds

16 months agoAdded new 'fail-requested' job status
Sybren A. Stüvel [Tue, 12 Feb 2019 16:06:56 +0000 (17:06 +0100)]
Added new 'fail-requested' job status

Instead of failing a job immediately, it can now also go to
'fail-requested' status. This should be done when there are still tasks
that are managed by the Manager (like 'active', 'claimed-by-manager', etc.)
and that will go to 'cancel-requested' rather than 'canceled'.

After a task has gone to 'canceled' status and when there are no more
'cancel-requested' tasks, the job will go from 'fail-requested' to 'failed'.

16 months agoAdded job['status_reason'] field for indicating reason for current status
Sybren A. Stüvel [Tue, 12 Feb 2019 16:03:29 +0000 (17:03 +0100)]
Added job['status_reason'] field for indicating reason for current status

This field is for human consumption, and is intended for indicating the
reason a job obtained its current status ("failed because X tasks failed",
"cancelled by request of user X", etc.).

Currently only the "failed job because too many tasks failed" status change
sets this field.

16 months agoAdded 'parent' link to tasks to browse to its parent(s)
Sybren A. Stüvel [Tue, 12 Feb 2019 15:03:25 +0000 (16:03 +0100)]
Added 'parent' link to tasks to browse to its parent(s)

16 months agoMerge remote-tracking branch 'origin/production'
Tobias Johansson [Tue, 12 Feb 2019 08:58:58 +0000 (09:58 +0100)]
Merge remote-tracking branch 'origin/production'