46 hours agoBetter readability of progressive render tasks master production
Sybren A. Stüvel [Thu, 14 Feb 2019 14:49:12 +0000 (15:49 +0100)]
Better readability of progressive render tasks

`render-smplA-B-frmX-Y` → `render-s_A-B-f_X-Y`
Same with merge tasks.

2 days agoIntroduced task status 'soft-failed'
Sybren A. Stüvel [Thu, 14 Feb 2019 10:29:11 +0000 (11:29 +0100)]
Introduced task status 'soft-failed'

This status indicates that there was a failure, but that the task will be
re-tried by another worker. As such, it should be handled fairly similarly
to 'claimed-by-manager'. Soft-failed tasks shouldn't affect the job status.

2 days agoAdded missing CSS colours for fail-requested status
Sybren A. Stüvel [Thu, 14 Feb 2019 10:27:11 +0000 (11:27 +0100)]
Added missing CSS colours for fail-requested status

2 days agoJob archival: don't crash on tasks without log
Sybren A. Stüvel [Thu, 14 Feb 2019 09:26:47 +0000 (10:26 +0100)]
Job archival: don't crash on tasks without log

2 days agoRunnability check: ensure tasks are counted only once
Sybren A. Stüvel [Wed, 13 Feb 2019 14:48:23 +0000 (15:48 +0100)]
Runnability check: ensure tasks are counted only once

Before this fix, each task was counted for each of its failed parents (so
a task with 3 failed parents would be counted 3 times).

2 days ago'fail-requested' job should stay 'fail-requested' even when tasks are active
Sybren A. Stüvel [Wed, 13 Feb 2019 14:35:43 +0000 (15:35 +0100)]
'fail-requested' job should stay 'fail-requested' even when tasks are active

Before, when a task update with status='active' would come in for a job
with status='fail-requested', the job status would be set to 'active'.
This is fixed now.

2 days agoVendoring Bowser rather than copying it from node_modules
Sybren A. Stüvel [Wed, 13 Feb 2019 13:55:25 +0000 (14:55 +0100)]
Vendoring Bowser rather than copying it from node_modules

It seems to work fine, except during staging for building the docker image;
then it silently fails to copy the bowser.js file into Flamenco's `static`

2 days agoJob view: show user and creation timestamp more explicitly
Sybren A. Stüvel [Wed, 13 Feb 2019 13:28:55 +0000 (14:28 +0100)]
Job view: show user and creation timestamp more explicitly

Previously only the username was shown (no full name), and the creation
timestamp was hidden behind the updated timestamp.

2 days agoAllow users to copy-paste job settings
Sybren A. Stüvel [Wed, 13 Feb 2019 13:22:40 +0000 (14:22 +0100)]
Allow users to copy-paste job settings

Before copying, variables in the job setting value are replaced based on
the user's own platform. Furthermore, specific settings can be processed
before copying; for example, the `render_output` setting gets its `######`
removed and a directory popped so that you end up at the directory
containing the intermediate directory and/or the final directory.

3 days agoRegression Fix: Choose flamenco manager broke while unifying ui.
Tobias Johansson [Wed, 13 Feb 2019 09:29:17 +0000 (10:29 +0100)]
Regression Fix: Choose flamenco manager broke while unifying ui.

Found using sentry.

3 days agoRegression Fix: Overflow in column scroll. Updated ugly hack.
Tobias Johansson [Wed, 13 Feb 2019 08:19:28 +0000 (09:19 +0100)]
Regression Fix: Overflow in column scroll. Updated ugly hack.

3 days agoSet job's 'status_reason' when recreating job
Sybren A. Stüvel [Tue, 12 Feb 2019 16:59:11 +0000 (17:59 +0100)]
Set job's 'status_reason' when recreating job

When recreating a job, its 'status_reason' field should be set,
otherwise it would keep it from the old run (and be meaningless at

3 days agoFix T50572 Detect unrunnable active jobs
Sybren A. Stüvel [Tue, 12 Feb 2019 16:58:57 +0000 (17:58 +0100)]
Fix T50572 Detect unrunnable active jobs

Active Flamenco jobs are periodically checked (via Celery) for non-runnable
tasks (e.g. queued tasks with a failed/cancelled parent). When such a task
is found, the entire job is failed (because it won't be able to complete

Add something like this to the CELERY_BEAT_SCHEDULE setting:

    'job-runnability-check': {
        'task': 'flamenco.celery.job_runnability_check.schedule_checks',
        'schedule': 600,  # every N seconds

3 days agoAdded new 'fail-requested' job status
Sybren A. Stüvel [Tue, 12 Feb 2019 16:06:56 +0000 (17:06 +0100)]
Added new 'fail-requested' job status

Instead of failing a job immediately, it can now also go to
'fail-requested' status. This should be done when there are still tasks
that are managed by the Manager (like 'active', 'claimed-by-manager', etc.)
and that will go to 'cancel-requested' rather than 'canceled'.

After a task has gone to 'canceled' status and when there are no more
'cancel-requested' tasks, the job will go from 'fail-requested' to 'failed'.

3 days agoAdded job['status_reason'] field for indicating reason for current status
Sybren A. Stüvel [Tue, 12 Feb 2019 16:03:29 +0000 (17:03 +0100)]
Added job['status_reason'] field for indicating reason for current status

This field is for human consumption, and is intended for indicating the
reason a job obtained its current status ("failed because X tasks failed",
"cancelled by request of user X", etc.).

Currently only the "failed job because too many tasks failed" status change
sets this field.

3 days agoAdded 'parent' link to tasks to browse to its parent(s)
Sybren A. Stüvel [Tue, 12 Feb 2019 15:03:25 +0000 (16:03 +0100)]
Added 'parent' link to tasks to browse to its parent(s)

4 days agoMerge remote-tracking branch 'origin/production'
Tobias Johansson [Tue, 12 Feb 2019 08:58:58 +0000 (09:58 +0100)]
Merge remote-tracking branch 'origin/production'

8 days agoUI: Slimmer entries and styling
Pablo Vazquez [Fri, 8 Feb 2019 01:56:00 +0000 (02:56 +0100)]
UI: Slimmer entries and styling

* no-margin between status cells.
* more prominent mouse-hover style.
* Less contrast on label for completed status (normal on mouse-hover).

8 days agoUI: Fix flamenco box non-white
Pablo Vazquez [Fri, 8 Feb 2019 01:54:33 +0000 (02:54 +0100)]
UI: Fix flamenco box non-white

8 days agoUI: Fix dropdown menu text overflow
Pablo Vazquez [Fri, 8 Feb 2019 01:54:17 +0000 (02:54 +0100)]
UI: Fix dropdown menu text overflow

8 days agoProgressive render: Non-render tasks get task_priority=1
Sybren A. Stüvel [Thu, 7 Feb 2019 17:13:47 +0000 (18:13 +0100)]
Progressive render: Non-render tasks get task_priority=1

This ensures that, given equal job priority, the simpler stuff (like
moving files around, or generating previews) takes priority over
rendering tasks.

8 days agoNavigation: Unified cloud navigation
Tobias Johansson [Thu, 7 Feb 2019 13:45:55 +0000 (14:45 +0100)]
Navigation: Unified cloud navigation

* Removed main drop down menu
* Added "My cloud" to user menu
* Attract/Flamenco is found under Production Tools menu
* Attract/Flamenco has the same navigation as its project

10 days agoNavigation: Unified cloud navigation
Tobias Johansson [Wed, 6 Feb 2019 09:31:36 +0000 (10:31 +0100)]
Navigation: Unified cloud navigation

* Removed main drop down menu
* Added "My cloud" to user menu
* Attract/Flamenco is found under Production Tools menu
* Attract/Flamenco has the same navigation as its project

10 days agoDepsgraph: on clicking a task, just reload the page
Sybren A. Stüvel [Wed, 6 Feb 2019 09:05:00 +0000 (10:05 +0100)]
Depsgraph: on clicking a task, just reload the page

This makes it clearer which nodes are correct and which are incomplete,
even though it's a bit more cumbersome and jumpy.

10 days agoDepsgraph: removed size warning
Sybren A. Stüvel [Wed, 6 Feb 2019 09:04:34 +0000 (10:04 +0100)]
Depsgraph: removed size warning

We don't draw in different ways using the Dagre-D3 library (it's always
drawing "nice").

10 days agoDepsgraph: don't show 3rd level nodes
Sybren A. Stüvel [Wed, 6 Feb 2019 09:03:24 +0000 (10:03 +0100)]
Depsgraph: don't show 3rd level nodes

It is still too slow, also for the new drawing library.

10 days agoProgressive rendering: dynamically set frame chunk based on max samples/task
Sybren A. Stüvel [Tue, 5 Feb 2019 16:53:48 +0000 (17:53 +0100)]
Progressive rendering: dynamically set frame chunk based on max samples/task

The frame chunk size in the job settings is multiplied with the sample
cap to obtain a maximum number of samples per render task. This is then
used to compute the number of frames that can be rendered in one render

10 days agoReplace Cytoscape with Dagre-D3 for depsgraph drawing
Sybren A. Stüvel [Tue, 5 Feb 2019 14:21:00 +0000 (15:21 +0100)]
Replace Cytoscape with Dagre-D3 for depsgraph drawing

It's not 100% feature-complete yet (status colour isn't drawn), but the
graph itself is much better organised.

10 days agoProgressive Rendering: merging after sample chunk now happens in one task
Sybren A. Stüvel [Tue, 5 Feb 2019 14:20:09 +0000 (15:20 +0100)]
Progressive Rendering: merging after sample chunk now happens in one task

This makes the work more efficient, as Blender is only started once (instead
of for every frame). It also removes the requirement that the frame chunks
are uniform, allowing for future optimisations.

11 days agoSplit test of blender_render_progressive into its own test file
Sybren A. Stüvel [Tue, 5 Feb 2019 12:09:02 +0000 (13:09 +0100)]
Split test of blender_render_progressive into its own test file

2 weeks agoAdded menu item for accessing JSON definition of tasks
Sybren A. Stüvel [Fri, 1 Feb 2019 09:58:02 +0000 (10:58 +0100)]
Added menu item for accessing JSON definition of tasks

2 weeks agoProgressive renders: only publish to out dir once full render is done
Sybren A. Stüvel [Fri, 1 Feb 2019 08:57:59 +0000 (09:57 +0100)]
Progressive renders: only publish to out dir once full render is done

This prevents any previous high-quality render from being overwritten
with the low-quality iterations of the current render job.

Requires Flamenco Worker de9204abfcad4bcb68eb3492d2e512de025a8bb8 or

2 weeks agoProgressive rendering: generate preview JPEG and MKV files
Sybren A. Stüvel [Thu, 31 Jan 2019 12:46:03 +0000 (13:46 +0100)]
Progressive rendering: generate preview JPEG and MKV files

After every sample chunk the EXR files are converted to JPEG and those are
then fed to FFmpeg to convert to an MKV video file.

2 weeks agoMore statuses in COLOR_FOR_TASK_STATUS
Sybren A. Stüvel [Thu, 31 Jan 2019 12:43:09 +0000 (13:43 +0100)]
More statuses in COLOR_FOR_TASK_STATUS

Also it's now a defaultdict, so that it doesn't raise a `KeyError` for
an unknown status.

2 weeks agoAllow job recreation in 'paused' and 'construction-failed' states
Sybren A. Stüvel [Thu, 31 Jan 2019 10:28:37 +0000 (11:28 +0100)]
Allow job recreation in 'paused' and 'construction-failed' states

The former is trivial, as in paused state the job isn't even sent to the
Manager. The latter is a newly-introduced state that allows differentiating
between 'actually constructing' and 'construction failed'. Previously the
job would simply hang in 'under-construction' state and not allow any

2 weeks agoProgressive render: convert EXR to JPEG + MKV previews
Sybren A. Stüvel [Thu, 31 Jan 2019 10:17:31 +0000 (11:17 +0100)]
Progressive render: convert EXR to JPEG + MKV previews

This is currently only done after the first sample chunk; this will be
addressed in another commit.

2 weeks agoProgressive rendering: nonuniform sample chunks
Sybren A. Stüvel [Wed, 30 Jan 2019 11:53:05 +0000 (12:53 +0100)]
Progressive rendering: nonuniform sample chunks

Sample chunks are no longer uniform, but initially start out small
(1/40th of the total sample count) and increase until an artist-provided
maximum sample count per task.

This is a backward-incompatible change, and requires Blender 2.80 beta
7744203b7fde3 or newer (from Tue Jan 29 18:08:12 2019 +0100), and the
latest Blender Cloud add-on and Flamenco Worker.

This is part of T50977.

2 weeks agoRemoved unused imports
Sybren A. Stüvel [Tue, 29 Jan 2019 13:07:31 +0000 (14:07 +0100)]
Removed unused imports

2 weeks agoStricter type check on `job` parameter in job compilers
Sybren A. Stüvel [Tue, 29 Jan 2019 12:45:40 +0000 (13:45 +0100)]
Stricter type check on `job` parameter in job compilers

Previously the code also would accept a PillarSDK Resource, but since that's
not what's in use at all, I removed that capability and just demand it's
a dict.

2 weeks agoHTML encoding fix for link URL in JavaScript
Sybren A. Stüvel [Tue, 29 Jan 2019 11:29:53 +0000 (12:29 +0100)]
HTML encoding fix for link URL in JavaScript

I also changed which part of the text is the link, as links should typically
be a noun and not a verb (https://www.w3.org/QA/Tips/noClickHere).

3 weeks agoTweak toastr message according to rF668cdccc5bac
Francesco Siddi [Sat, 19 Jan 2019 18:27:11 +0000 (19:27 +0100)]
Tweak toastr message according to rF668cdccc5bac

We expose the URL where to find the task log within the notification, before we attempt to
open it in another window.

5 weeks agoAllow 'Requeue task and successors' when the task itself is already 'queued'
Sybren A. Stüvel [Thu, 10 Jan 2019 10:43:21 +0000 (11:43 +0100)]
Allow 'Requeue task and successors' when the task itself is already 'queued'

This is useful after editing RNA overrides. The RNA override task itself
will be queued then, and the 'Requeue task and successors' button will now
show to allow re-queueing all render tasks.

5 weeks agoRNA Overrides: added validation of Python code
Sybren A. Stüvel [Thu, 10 Jan 2019 10:24:00 +0000 (11:24 +0100)]
RNA Overrides: added validation of Python code

This validation at least parses the override's Python code, which should
catch all syntax errors. Of course semantic errors are still possible.

5 weeks agoAllow setting RNA overrides via web interface
Sybren A. Stüvel [Fri, 4 Jan 2019 14:25:21 +0000 (15:25 +0100)]
Allow setting RNA overrides via web interface

The interface is very simple, but good enough to test the feature.

5 weeks agoImplemented PATCH of job to create/update RNA overrides
Sybren A. Stüvel [Thu, 3 Jan 2019 18:08:12 +0000 (19:08 +0100)]
Implemented PATCH of job to create/update RNA overrides

5 weeks agoCreate an RNA override file for Blender renders
Sybren A. Stüvel [Thu, 3 Jan 2019 16:27:23 +0000 (17:27 +0100)]
Create an RNA override file for Blender renders

When the job settings contains a key `'rna_overrides'` it is assumed to
be a list of strings. Those strings are concatenated into a Python file,
which should be loaded after the blend file was loaded (and before actual
rendering starts).

Does not contain any way to set those settings yet, apart from mucking
around in the MongoDB directly.

5 weeks agoTest Blender render jobs with filepath ending in .blend
Sybren A. Stüvel [Fri, 4 Jan 2019 12:49:54 +0000 (13:49 +0100)]
Test Blender render jobs with filepath ending in .blend

5 weeks agoSimplified some JavaScript
Sybren A. Stüvel [Fri, 4 Jan 2019 14:24:51 +0000 (15:24 +0100)]
Simplified some JavaScript

5 weeks agoLog upload requests: also allow when task is in state claimed-by-manager
Sybren A. Stüvel [Thu, 10 Jan 2019 09:47:50 +0000 (10:47 +0100)]
Log upload requests: also allow when task is in state claimed-by-manager

This is the state a task returns to when a worker signs off and hands the
task back to the Manager for requeueing. When that happens, the log should
remain requestable.

5 weeks agoAllow users to request task log files from Manager
Sybren A. Stüvel [Tue, 8 Jan 2019 13:50:32 +0000 (14:50 +0100)]
Allow users to request task log files from Manager

Nowadays the Manager keeps the log files, and the Server only stores the
last few lines of logging; the entire log files can be viewed on the
Manager directly. This is fine, unless a user doesn't have access to the
Manager itself.

This commit allows users to request the Manager to send its logfile to
the Server. These requests are queued, and the next time the Manager
sends task updates it receives those requests and acts accordingly.

Uploaded log files are compressed by the Manager, and stored as-is on
the project's storage backend until the task is requeued & reclaimed by
the Manager (in which case we act as if it's gone, but the file is still
there; it's just the reference that's erased).

The task's logfile can only be requested in certain states (all except
queued, claimed-by-manager, and active). This prevents a task file from
being requested while the task is still active (and thus this feature
misused for realtime updates).

5 weeks agoRemoved local imports
Sybren A. Stüvel [Tue, 8 Jan 2019 14:24:58 +0000 (15:24 +0100)]
Removed local imports

5 weeks agoList task type (not job type) in the task list
Sybren A. Stüvel [Tue, 8 Jan 2019 13:47:50 +0000 (14:47 +0100)]
List task type (not job type) in the task list

5 weeks agoFix sorting of tasks
Sybren A. Stüvel [Fri, 4 Jan 2019 14:24:34 +0000 (15:24 +0100)]
Fix sorting of tasks

5 weeks agoGulp fix, run 'npm install natives@1.1.6' too
Sybren A. Stüvel [Fri, 4 Jan 2019 13:19:33 +0000 (14:19 +0100)]
Gulp fix, run 'npm install natives@1.1.6' too

5 weeks agoPrevent key error when validating settings of not-yet-saved job
Sybren A. Stüvel [Fri, 4 Jan 2019 12:49:33 +0000 (13:49 +0100)]
Prevent key error when validating settings of not-yet-saved job

5 weeks agoCorrected logger name
Sybren A. Stüvel [Fri, 4 Jan 2019 12:49:05 +0000 (13:49 +0100)]
Corrected logger name

5 weeks agoAdded docstring
Sybren A. Stüvel [Tue, 8 Jan 2019 13:48:50 +0000 (14:48 +0100)]
Added docstring

5 weeks agoFixed error message in manager_api_call()
Sybren A. Stüvel [Tue, 8 Jan 2019 13:48:34 +0000 (14:48 +0100)]
Fixed error message in manager_api_call()

6 weeks agoMoved local imports to module level
Sybren A. Stüvel [Thu, 3 Jan 2019 13:19:11 +0000 (14:19 +0100)]
Moved local imports to module level

8 weeks agoFix T55434: Confusing error when accessing real job while logged out
Sybren A. Stüvel [Thu, 20 Dec 2018 12:12:28 +0000 (13:12 +0100)]
Fix T55434: Confusing error when accessing real job while logged out

8 weeks agoFix infinite recursion when getting job beyond first page of results
Sybren A. Stüvel [Thu, 20 Dec 2018 10:53:42 +0000 (11:53 +0100)]
Fix infinite recursion when getting job beyond first page of results

The old code acted as if the job was archived when it wasn't returned by
the query. This is somewhat correct for the first page, but not for later

8 weeks agoAllow admins to create an infinite number of Managers
Sybren A. Stüvel [Thu, 20 Dec 2018 10:26:10 +0000 (11:26 +0100)]
Allow admins to create an infinite number of Managers

8 weeks agoLimit list of Managers to owned Managers
Sybren A. Stüvel [Thu, 20 Dec 2018 10:25:58 +0000 (11:25 +0100)]
Limit list of Managers to owned Managers

Previously admins would see all managers on the system, which was too long
a list to be useful.

8 weeks agoLimit /flamenco to projects current user is member of
Sybren A. Stüvel [Thu, 20 Dec 2018 10:18:02 +0000 (11:18 +0100)]
Limit /flamenco to projects current user is member of

This is in contrast to the old behaviour, which listed all projects the
user had access to (which is all projects for admins).

8 weeks agoRemoved noisy debug log
Sybren A. Stüvel [Tue, 18 Dec 2018 14:06:24 +0000 (15:06 +0100)]
Removed noisy debug log

8 weeks agoFix T51803: allow cancellation of individual tasks
Sybren A. Stüvel [Tue, 18 Dec 2018 11:02:21 +0000 (12:02 +0100)]
Fix T51803: allow cancellation of individual tasks

This required a little change in the task status → job status propagation.
The code assumed that a task going from `cancel_requested` → `canceled`
was part of the cancellation of an entire job. This no longer is the
case, so now the job is only cancelled if it was `cancel_requested` before.

2 months agoVue Comments: Comments ported to Vue + DnD fileupload
Tobias Johansson [Wed, 12 Dec 2018 10:45:46 +0000 (11:45 +0100)]
Vue Comments: Comments ported to Vue + DnD fileupload

* Drag and drop files to comment editor to add a file attachment
* Using Vue to render comments

Since comments now has attachments we need to update the schemas
./manage.py maintenance replace_pillar_node_type_schemas

2 months agoRequeue task + successors
Sybren A. Stüvel [Fri, 7 Dec 2018 13:05:54 +0000 (14:05 +0100)]
Requeue task + successors

Allow requeueing a task with all its successors (e.g. those tasks that are
dependent on the requeued one and thus need re-execution).

2 months agoAdded missing `ffmpeg_cmd` parameter in certain commands
Sybren A. Stüvel [Fri, 7 Dec 2018 12:43:02 +0000 (13:43 +0100)]
Added missing `ffmpeg_cmd` parameter in certain commands

This is required for the Manager to do proper variable replacement.

2 months agoWhen recreating a job, redirect to the job and not one of the old tasks
Sybren A. Stüvel [Fri, 7 Dec 2018 10:10:46 +0000 (11:10 +0100)]
When recreating a job, redirect to the job and not one of the old tasks

Just reloading the current URL can lead to `../with-task/{old task ID}`,
which of course doesn't exist any more when the job is being recreated.

2 months agoAdded `blender-video-chunks` job type
Sybren A. Stüvel [Thu, 6 Dec 2018 14:49:38 +0000 (15:49 +0100)]
Added `blender-video-chunks` job type

This job type is meant for encoding the edit of a film from the video
sequence editor. In the Blender Cloud add-on, configure it to use
Matroska video and be sure to configure an audio codec if you want audio
to be in the final file.

The job consists of the following tasks:
    - Per video chunk, render the frames as PNG and combine each chunk
      into a Matroska (MKV) video file.
    - Concatenate all MKV files into one file.
    - Render the audio, convert to AAC.
    - Mux audio and video into a single MKV file.
    - Move that file to the final location, and name it with the current
      date, a counter to make the filename unique, and the name of the
      blend file.

Requires Blender Cloud add-on 2.0 or newer, and Flamenco Worker 2.2 or

2 months agoFixed formatting mistake
Sybren A. Stüvel [Thu, 6 Dec 2018 14:49:11 +0000 (15:49 +0100)]
Fixed formatting mistake

2 months agoRemoved unused code
Sybren A. Stüvel [Tue, 4 Dec 2018 16:19:08 +0000 (17:19 +0100)]
Removed unused code

2 months agoAdded 'exec-command' job for debugging purposes
Sybren A. Stüvel [Tue, 4 Dec 2018 16:19:02 +0000 (17:19 +0100)]
Added 'exec-command' job for debugging purposes

2 months agoShow "View log on Manager" button when we suspect it's available there v2.1
Sybren A. Stüvel [Tue, 4 Dec 2018 10:29:13 +0000 (11:29 +0100)]
Show "View log on Manager" button when we suspect it's available there

2 months agoBumped version to 2.1
Sybren A. Stüvel [Tue, 4 Dec 2018 09:39:56 +0000 (10:39 +0100)]
Bumped version to 2.1

2 months agoUpdated CHANGELOG
Sybren A. Stüvel [Tue, 4 Dec 2018 09:39:18 +0000 (10:39 +0100)]

2 months agoShow last log lines in the view-task-embed
Sybren A. Stüvel [Fri, 30 Nov 2018 16:54:19 +0000 (17:54 +0100)]
Show last log lines in the view-task-embed

2 months agoRevert "Workaround hack for long archive jobs blocking the Celery worker"
Sybren A. Stüvel [Tue, 4 Dec 2018 09:13:48 +0000 (10:13 +0100)]
Revert "Workaround hack for long archive jobs blocking the Celery worker"

This reverts commit 5083076d896d50fb9e0c513765447ab3e0fe593a.

The bug that we wanted to fix is worked around by switching from RabbitMQ
to Redis as Celery Broker.

2 months agoWorkaround hack for long archive jobs blocking the Celery worker
Sybren A. Stüvel [Fri, 30 Nov 2018 15:32:34 +0000 (16:32 +0100)]
Workaround hack for long archive jobs blocking the Celery worker

This is due to a Celery bug (https://github.com/celery/celery/issues/3430).
As a workaround we now execute the tasks (so that they're marked as
completed in the Celery queue) but don't actually perform the functionality.
A proper fix will be worked on next Tuesday.

2 months agoTask logs: corrected index & search query
Sybren A. Stüvel [Thu, 29 Nov 2018 12:39:13 +0000 (13:39 +0100)]
Task logs: corrected index & search query

Two mistakes:

- The index used the field `task_id`, which doesn't exist (it's `task`)
- The job archival query used slightly different sort arguments, so it
  didn't match the index.

2 months agoUpdated changelog
Sybren A. Stüvel [Thu, 29 Nov 2018 10:36:25 +0000 (11:36 +0100)]
Updated changelog

2 months agoRe-queue a job if its last task gets re-queued by the Manager
Sybren A. Stüvel [Thu, 29 Nov 2018 10:16:00 +0000 (11:16 +0100)]
Re-queue a job if its last task gets re-queued by the Manager

2 months agoAllow users to change the priority of a job
Sybren A. Stüvel [Wed, 28 Nov 2018 14:51:15 +0000 (15:51 +0100)]
Allow users to change the priority of a job

2 months agoAllow PATCH {'op': 'set-job-priority'} on jobs to change their priority
Sybren A. Stüvel [Wed, 28 Nov 2018 14:31:59 +0000 (15:31 +0100)]
Allow PATCH {'op': 'set-job-priority'} on jobs to change their priority

Tasks of the job will also be changed to reflect the new job priority,
and will be sent to the Manager.

2 months agoJob archival: ignore already-archived and non-existing jobs
Sybren A. Stüvel [Wed, 28 Nov 2018 11:48:51 +0000 (12:48 +0100)]
Job archival: ignore already-archived and non-existing jobs

The `resume_job_archiving()` Celery task can queue a large number of archival
jobs. The total time needed for all that archiving can take longer than
the scheduled period of the `resume_job_archiving()` task, causing those
archival tasks to be scheduled again. These 2nd archival tasks should
not break anything, so checking whether a job is already archived seems
like the best approach to me.

2 months agoFormatting
Sybren A. Stüvel [Wed, 28 Nov 2018 11:46:34 +0000 (12:46 +0100)]

2 months agoAdded note about getting ANSI colours on Windows
Sybren A. Stüvel [Wed, 28 Nov 2018 08:56:33 +0000 (09:56 +0100)]
Added note about getting ANSI colours on Windows

2 months agoAdded video encoding requirements to the installation manual
Sybren A. Stüvel [Fri, 23 Nov 2018 16:11:58 +0000 (17:11 +0100)]
Added video encoding requirements to the installation manual

2 months agoUse `pipenv` in `deploy_docs.sh`
Sybren A. Stüvel [Wed, 28 Nov 2018 08:56:49 +0000 (09:56 +0100)]
Use `pipenv` in `deploy_docs.sh`

2 months agoAdded Pipfile for docs
Sybren A. Stüvel [Fri, 23 Nov 2018 16:14:35 +0000 (17:14 +0100)]
Added Pipfile for docs

The `mkdocs` command seems to require another version of MarkDown than we
use in Pillar, so separating the virtualenvs seemed like the easiest way
to go.

2 months agoInclude `ffmpeg_cmd: {ffmpeg}` variable in create-video tasks
Sybren A. Stüvel [Fri, 23 Nov 2018 14:40:07 +0000 (15:40 +0100)]
Include `ffmpeg_cmd: {ffmpeg}` variable in create-video tasks

This allows the Manager to determine which FFmpeg binary is used. We were
thinking of bundling FFmpeg with the worker, but it turns out that the
binaries for different platforms may or may not have high-bit video support
(it's not documented well), so we wouldn't know exactly what we'd be
bundling. This made me decide against it (at least for now) so the user
will have to install FFmpeg and configure its binary location.

2 months agoFix showing task types when never sent by worker
Sybren A. Stüvel [Fri, 23 Nov 2018 14:38:34 +0000 (15:38 +0100)]
Fix showing task types when never sent by worker

Apparently the Jinja `sort` filter crashes when the input is None.

2 months agoMerge branch 'production'
Sybren A. Stüvel [Thu, 22 Nov 2018 15:06:12 +0000 (16:06 +0100)]
Merge branch 'production'

2 months agoInsert create-video task when the job has the appropriate settings
Sybren A. Stüvel [Wed, 21 Nov 2018 14:10:59 +0000 (15:10 +0100)]
Insert create-video task when the job has the appropriate settings

Include these settings to enable that task:

- image_or_video: 'images'
- output_file_extension: anything like '.png' or '.exr', needs to match the
  files produced by the job
- fps: any integer or float, desired frame rate of the video

Only when the Manager declares it has at least one worker that can run
tasks of type 'video-encoding' will we create a 'create-video' task for
'blender-render' jobs. This is checked in the Manager document
'worker_task_types' key.

2 months agoFlamenco: Receive set of worker task types from the Manager
Sybren A. Stüvel [Thu, 22 Nov 2018 12:41:56 +0000 (13:41 +0100)]
Flamenco: Receive set of worker task types from the Manager

Knowing which task types are supported by the Manager's Workers will allow
us to tailor job compilation for their capabilities. For example, we should
only include 'create-video' tasks in a render job when there actually is
a worker that can execute 'video-encoding' tasks.

2 months agoShow last-assigned worker name in task list
Sybren A. Stüvel [Wed, 21 Nov 2018 14:36:45 +0000 (15:36 +0100)]
Show last-assigned worker name in task list

2 months agoRemoved obsolete TODOs
Sybren A. Stüvel [Wed, 21 Nov 2018 14:16:30 +0000 (15:16 +0100)]
Removed obsolete TODOs

2 months agoAdded MyPy configuration
Sybren A. Stüvel [Wed, 21 Nov 2018 14:13:25 +0000 (15:13 +0100)]
Added MyPy configuration

There are still 30 errors reported.

2 months agoUse @attr.s(auto_attribs=True) in command definitions
Sybren A. Stüvel [Wed, 21 Nov 2018 14:09:37 +0000 (15:09 +0100)]
Use @attr.s(auto_attribs=True) in command definitions