31 hours agoHandle connection errors on a restart request as OK master
Sybren A. Stüvel [Wed, 24 Apr 2019 13:54:13 +0000 (15:54 +0200)]
Handle connection errors on a restart request as OK

32 hours agoSend '204 No Content' when restarting
Sybren A. Stüvel [Wed, 24 Apr 2019 13:21:28 +0000 (15:21 +0200)]
Send '204 No Content' when restarting

This prevents the browser from getting a 'connection lost' error.

6 days agoAutomatic TLS certificate'ing with Let's Encrypt
Sybren A. Stüvel [Fri, 19 Apr 2019 11:53:32 +0000 (13:53 +0200)]
Automatic TLS certificate'ing with Let's Encrypt

To use Let's Encrypt:
- Set `acme_domain_name` to the domain name of the machine.
- Set both `listen` and `listen_https` to the ports Flamenco Manager
  should be listening to. By default these are `:8080` and `:8443`.
- Configure your firewall or user-facing proxy to forward ports 80 and
  443 to respectively 8080 and 8443.

Other changes:
- Added setting `listen_https` which is used for serving HTTPS traffic
  (default `:8443`). If you are using the `tlskey`/`tlscert` settings, you
  need to move `listen` to `listen_https`.
- Changed the default value for `listen` to `:8080` (was `:8083`).

The changes to the default were somewhat necessary to get to more
standard port numbers; it would be silly to add the standard port number
8443 and still keep using the nonstandard 8083.

A new webserver wrapper was introduced that manages both the HTTP and
HTTPS servers as a single unit. When using ACME/Let's Encrypt it is
necessary to have both HTTP (for the ACME web authentication) and HTTPS
(for regular traffic). All other HTTP traffic is redirected to HTTPS on
port 443. This does *not* redirect to the configured `listen_https` port
because firewall-based redirection or reverse proxies may be in use.
Actually, this is recommended because then Flamenco Manager doesn't need
to be run as root.

6 days agoReduce log level
Sybren A. Stüvel [Fri, 19 Apr 2019 11:40:03 +0000 (13:40 +0200)]
Reduce log level

6 days agoJust a comment to clarify `own_url`
Sybren A. Stüvel [Fri, 19 Apr 2019 11:39:54 +0000 (13:39 +0200)]
Just a comment to clarify `own_url`

6 days agoREADME: removed `-verbose` and added `-quiet`
Sybren A. Stüvel [Fri, 19 Apr 2019 10:57:16 +0000 (12:57 +0200)]
README: removed `-verbose` and added `-quiet`

6 days agoMakefile: allow passing build options in BUILDOPTS
Sybren A. Stüvel [Fri, 19 Apr 2019 10:57:02 +0000 (12:57 +0200)]
Makefile: allow passing build options in BUILDOPTS

7 days agoMerge branch 'hotfix-v2.4'
Sybren A. Stüvel [Thu, 18 Apr 2019 15:48:25 +0000 (17:48 +0200)]
Merge branch 'hotfix-v2.4'

7 days agoCall `magick.exe convert` on Windows, instead of `convert`
Sybren A. Stüvel [Thu, 18 Apr 2019 15:43:16 +0000 (17:43 +0200)]
Call `magick.exe convert` on Windows, instead of `convert`

`convert` is the filesystem conversion tool, not the image conversion tool.

7 days agoFixed warning
Sybren A. Stüvel [Thu, 18 Apr 2019 15:34:35 +0000 (17:34 +0200)]
Fixed warning

7 days agoSettings version 2
Sybren A. Stüvel [Tue, 26 Mar 2019 09:45:27 +0000 (10:45 +0100)]
Settings version 2

Settings are now versioned; unversioned settings are implied to be
version 1. This commit introduces settings version 2, which changes the
way variables work. This requires Flamenco Server > 2.4.

- Path replacement and regular variables have been merged into one set
  of variables.
- Path replacement variables are now marked with `direction: twoway`,
  whereas what used to be regular variables are now marked with
  `direction: oneway`.
- Values can now have an audience, which is either 'all' (the old
  behaviour and implied when `audience=''`), 'workers' (only sent to
  workers), or 'users' (sent to Flamenco Server so that users see them
  in the web interface + Blender Cloud add-on).

Variables are automatically converted to version 2 when loading the
settings, even when the settings are still specifying the old style.
When saving the configuration file it is saved as version 2.

7 days agoBig refactor of web setup mode (JWT security + YAML editor + Vue.js)
Sybren A. Stüvel [Fri, 22 Mar 2019 13:05:32 +0000 (14:05 +0100)]
Big refactor of web setup mode (JWT security + YAML editor + Vue.js)

Web setup is now optionally secured with JWT tokens. When the Manager is
not linked to a Server, this SECURITY IS TURNED OFF. Flamenco Server is
used to supply us with JWT tokens and public keys for token validation;
without knowing which server to connect to, this workflow is impossible.

Other changes are:
- Immediately download JWT keys upon starting. This is most important
  for the setup mode, where we may need to download keys immediately
  after restarting (which happens after linking).
- Web setup now uses main layout.html file and Vue.js.
- Web setup now loads and saves settings via YAML. It can show a web
  form as well as an advanced mode that allows direct editing of YAML.

Note that the YAML is never sent byte-for-byte from the config file to
the web frontend. It is always parsed, modified (the `ManagerSecret`
property is cleared), and then re-marshalled to YAML before sending to
the frontend. The frontend does the same: it first parses the YAML from
the editor (showing a clear description about any parse errors), then
converts the parsed configuration back to YAML before sending it to the

7 days agoUpdated changelog
Sybren A. Stüvel [Thu, 18 Apr 2019 12:21:54 +0000 (14:21 +0200)]
Updated changelog

8 days agoGo to setup mode when Manager is not yet linked
Sybren A. Stüvel [Wed, 17 Apr 2019 15:51:11 +0000 (17:51 +0200)]
Go to setup mode when Manager is not yet linked

Even when there is a configuration file, if the Manager ID is empty it will
still be forced into Setup mode.

4 weeks agoBundled MongoDB: Connect to instead of localhost v2.4.2
Sybren A. Stüvel [Wed, 27 Mar 2019 10:01:44 +0000 (11:01 +0100)]
Bundled MongoDB: Connect to instead of localhost

On Windows, connecting to is much faster than to localhost for
some reason.

4 weeks agoLoop the connection attempt to MongoDB v2.4.1
Sybren A. Stüvel [Wed, 27 Mar 2019 09:47:49 +0000 (10:47 +0100)]
Loop the connection attempt to MongoDB

This allows the MongoDB server to start up very slowly, and still eventually
we'll have a connection.

4 weeks agoClarification of version numbers in README
Sybren A. Stüvel [Wed, 27 Mar 2019 09:20:15 +0000 (10:20 +0100)]
Clarification of version numbers in README

Now that we're about to release v2.4.1, the README should allow for
fix/micro numbers in the version number.

4 weeks agoSuggest using external MongoDB when installing on Windows
Sybren A. Stüvel [Wed, 27 Mar 2019 09:19:42 +0000 (10:19 +0100)]
Suggest using external MongoDB when installing on Windows

Either the bundled MongoDB, or the way we start it, seems to cause some
stability issues, and I don't have the resources to dive into this. Running
MongoDB externally from Flamenco Manager was found to be a proper solution
to the people who contacted me about this.

4 weeks agoCorrected `jwtkeys/README.md` file + included in dist package
Sybren A. Stüvel [Wed, 27 Mar 2019 09:07:51 +0000 (10:07 +0100)]
Corrected `jwtkeys/README.md` file + included in dist package

The info in the README was out of date, and it wasn't included in the
distribution package either.

4 weeks agoFixed 'The process cannot access the file' error on Windows
Sybren A. Stüvel [Wed, 27 Mar 2019 09:03:34 +0000 (10:03 +0100)]
Fixed 'The process cannot access the file' error on Windows

I forgot to close the file before renaming it.

4 weeks agoMake `jwtkeys` directory if it doesn't exist yet
Sybren A. Stüvel [Wed, 27 Mar 2019 09:02:56 +0000 (10:02 +0100)]
Make `jwtkeys` directory if it doesn't exist yet

4 weeks agoAdded explanation of version numbers to documentation
Sybren A. Stüvel [Tue, 26 Mar 2019 09:57:44 +0000 (10:57 +0100)]
Added explanation of version numbers to documentation

4 weeks agoUpdated README to reflect current reality
Sybren A. Stüvel [Tue, 26 Mar 2019 09:57:34 +0000 (10:57 +0100)]
Updated README to reflect current reality

4 weeks agoSimplified some JS
Sybren A. Stüvel [Tue, 26 Mar 2019 09:45:41 +0000 (10:45 +0100)]
Simplified some JS

4 weeks agoSimplified code a bit
Sybren A. Stüvel [Tue, 26 Mar 2019 09:45:35 +0000 (10:45 +0100)]
Simplified code a bit

4 weeks agoRemove 'v' from 'v2.4' version number in package file names v2.4
Sybren A. Stüvel [Mon, 25 Mar 2019 16:22:52 +0000 (17:22 +0100)]
Remove 'v' from 'v2.4' version number in package file names

4 weeks agoServe last-image.html as static page
Sybren A. Stüvel [Mon, 25 Mar 2019 14:56:43 +0000 (15:56 +0100)]
Serve last-image.html as static page

This makes it easier to handle JWT tokens.

4 weeks agoFixed VERSION in Makefile
Sybren A. Stüvel [Mon, 25 Mar 2019 15:38:53 +0000 (16:38 +0100)]
Fixed VERSION in Makefile

4 weeks agoMarked version 2.4 as released
Sybren A. Stüvel [Mon, 25 Mar 2019 15:34:57 +0000 (16:34 +0100)]
Marked version 2.4 as released

4 weeks agoAllow restarting Manager without auth when auth is disabled
Sybren A. Stüvel [Fri, 22 Mar 2019 13:04:26 +0000 (14:04 +0100)]
Allow restarting Manager without auth when auth is disabled

4 weeks agoClarified function name
Sybren A. Stüvel [Fri, 22 Mar 2019 12:36:48 +0000 (13:36 +0100)]
Clarified function name

5 weeks agoNever save JWT public keys URL, it's computed from the Flamenco Server URL
Sybren A. Stüvel [Thu, 21 Mar 2019 15:45:11 +0000 (16:45 +0100)]
Never save JWT public keys URL, it's computed from the Flamenco Server URL

5 weeks agoUpdated some default values for config settings
Sybren A. Stüvel [Thu, 21 Mar 2019 15:25:55 +0000 (16:25 +0100)]
Updated some default values for config settings

These values are what we've been successfully using at the Blender
Animation Studio during the production of Spring.

5 weeks agoSlight tweaks to restart pages
Sybren A. Stüvel [Thu, 21 Mar 2019 14:16:23 +0000 (15:16 +0100)]
Slight tweaks to restart pages

5 weeks agoRedirect /setup to / when in normal mode
Sybren A. Stüvel [Thu, 21 Mar 2019 14:16:14 +0000 (15:16 +0100)]
Redirect /setup to / when in normal mode

5 weeks agoUpdated CHANGELOG
Sybren A. Stüvel [Thu, 21 Mar 2019 14:09:08 +0000 (15:09 +0100)]

5 weeks agoAllow toggling between normal and setup mode from the web
Sybren A. Stüvel [Thu, 21 Mar 2019 14:06:49 +0000 (15:06 +0100)]
Allow toggling between normal and setup mode from the web

This makes it possible to enter setup, alter config, and restart into
normal mode, all from the web interface.

5 weeks agoWebsetup: removed 'advanced users' warning
Sybren A. Stüvel [Thu, 21 Mar 2019 14:06:10 +0000 (15:06 +0100)]
Websetup: removed 'advanced users' warning

The warning was about using systemd and the like, and no longer applies
since we use syscall.Exec() to restart the process in-place (since 4699c70).

5 weeks agoDashboard now also uses a layout.html
Sybren A. Stüvel [Thu, 21 Mar 2019 14:05:05 +0000 (15:05 +0100)]
Dashboard now also uses a layout.html

This means that we should probably merge the websetup and dashboard template
rendering code, because it's now simply a copy of each other.

5 weeks agoSort endpoints by alphabet
Sybren A. Stüvel [Thu, 21 Mar 2019 13:58:38 +0000 (14:58 +0100)]
Sort endpoints by alphabet

5 weeks agoMoved '/static/latest-image.jpg' endpoint to correct section
Sybren A. Stüvel [Thu, 21 Mar 2019 13:58:23 +0000 (14:58 +0100)]
Moved '/static/latest-image.jpg' endpoint to correct section

5 weeks agoTypo in Vue component HTML
Sybren A. Stüvel [Thu, 21 Mar 2019 13:53:06 +0000 (14:53 +0100)]
Typo in Vue component HTML

5 weeks agoRemoved LICENSE BLOCK lines + moved license block to top of file
Sybren A. Stüvel [Thu, 21 Mar 2019 13:06:10 +0000 (14:06 +0100)]
Removed LICENSE BLOCK lines + moved license block to top of file

The LICENSE BLOCK lines were there because Blender does that too, but they
are being/have been removed from the Blender sources.

Apparently when there is a newline between the comment and the `package`
keyword, it's no longer interpreted as package documentation (which is why
I had the comment below the `package` keyword in the first place).

5 weeks agoUse syscall.Exec() to restart on POSIX-compliant platforms
Sybren A. Stüvel [Thu, 21 Mar 2019 12:47:53 +0000 (13:47 +0100)]
Use syscall.Exec() to restart on POSIX-compliant platforms

Windows still uses the spawn-child-and-kill-parent approach.

5 weeks agoSetup mode: only temporarily override cliArgs.quiet
Sybren A. Stüvel [Thu, 21 Mar 2019 11:24:46 +0000 (12:24 +0100)]
Setup mode: only temporarily override cliArgs.quiet

This makes it possible to restore the user-requested quietness when
restarting into normal mode.

5 weeks agoBetter documentation of CLI args
Sybren A. Stüvel [Thu, 21 Mar 2019 11:24:14 +0000 (12:24 +0100)]
Better documentation of CLI args

5 weeks agoSplit main.go into several smaller files + renamed `httpendpoints.go`
Sybren A. Stüvel [Thu, 21 Mar 2019 10:46:48 +0000 (11:46 +0100)]
Split main.go into several smaller files + renamed `httpendpoints.go`

- `main.go`: entry point and HTTP server
- `normalmode.go`: service management + HTTP routes for normal operation
- `normalendpoints.go`: renamed from  `httpendpoints.go` to match
- `programstate.go`: CLI arguments, service pointers, and shutdown routine
- `setupmode.go`: HTTP routes for setup mode
- `restart.go`: code for handling restarting the process

There are no functional/semantic changes.

5 weeks agoMoved URL to end of Curl command
Sybren A. Stüvel [Tue, 19 Mar 2019 16:14:43 +0000 (17:14 +0100)]
Moved URL to end of Curl command

This makes it easier to copy the Curl command of one task, then edit the
URL to point at the task you actually want.
curl -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9.eyJhdWQiOiI1OWE4MjI2ZDk4Mzc3MzQ4YzM1YWYxNzciLCJleHAiOjE1NTMwMTIxMDIuNDY2NzA3LCJzdWIiOiI1ODBmOGM2Njk4Mzc3Mzc1OWFmZGIyMGUiLCJpYXQiOjE1NTMwMTIwNDIuNDY2NzA3fQ.58MMOCSJpLFZkf0QRsR9UjmCnFwtLqLxtTOvkV5kjtGi1v-F3YsgSQ1OFfpMx6DvjtCHhL6ZEDVAhs7C0GVbzQ" http://flamanager.local:8083/logfile/000000000000000000000000/5c90cbf8966305e9fd86eae8

5 weeks agoSecure worker registration with a JWT signed with a pre-shared secret
Sybren A. Stüvel [Tue, 19 Mar 2019 15:13:45 +0000 (16:13 +0100)]
Secure worker registration with a JWT signed with a pre-shared secret

Both the Workers and Manager need to have the same value for their
`worker_registration_secret` configuration setting (unless the Manager
has an empty string there, in which case registration is open to anyone).

The pre-shared secret is used by the Worker to create a short-lived JWT
token, which is then used in an `Authorization: Bearer {token}` header when
POSTing to the `/register-worker` endpoint.

5 weeks agoChange config key 'authentication' to 'user_authentication'
Sybren A. Stüvel [Tue, 19 Mar 2019 13:27:47 +0000 (14:27 +0100)]
Change config key 'authentication' to 'user_authentication'

5 weeks agoAdded toastr message to 'copy curl' button
Sybren A. Stüvel [Tue, 19 Mar 2019 13:07:16 +0000 (14:07 +0100)]
Added toastr message to 'copy curl' button

5 weeks agoNicer styling for 'view log' and 'copy curl'
Sybren A. Stüvel [Tue, 19 Mar 2019 13:04:54 +0000 (14:04 +0100)]
Nicer styling for 'view log' and 'copy curl'

5 weeks agoUpdated changelog
Sybren A. Stüvel [Tue, 19 Mar 2019 12:41:03 +0000 (13:41 +0100)]
Updated changelog

5 weeks agoDon't register `/jwt/token-urls` endpoint when authentication was disabled
Sybren A. Stüvel [Tue, 19 Mar 2019 12:39:22 +0000 (13:39 +0100)]
Don't register `/jwt/token-urls` endpoint when authentication was disabled

5 weeks agoAdded 'Copy Curl' to copy the Curl command needed to download a task log
Sybren A. Stüvel [Tue, 19 Mar 2019 12:38:51 +0000 (13:38 +0100)]
Added 'Copy Curl' to copy the Curl command needed to download a task log

This is a bit cumbersome as the Curl command needs an authentication token
to download the logs.

5 weeks agoCache JWT token in JS variable
Sybren A. Stüvel [Tue, 19 Mar 2019 11:44:51 +0000 (12:44 +0100)]
Cache JWT token in JS variable

This makes frequent calls to `jwtToken()` more efficient because that we
don't have to round-trip to cookies all the time.

5 weeks agoLog more elaborate warning when /get-token is called without private key loaded
Sybren A. Stüvel [Tue, 19 Mar 2019 11:18:10 +0000 (12:18 +0100)]
Log more elaborate warning when /get-token is called without private key loaded

5 weeks agoProtect /kick and task log endpoints with JWT tokens
Sybren A. Stüvel [Tue, 19 Mar 2019 11:09:39 +0000 (12:09 +0100)]
Protect /kick and task log endpoints with JWT tokens

5 weeks agoDashboard: only show worker list if there are workers
Sybren A. Stüvel [Tue, 19 Mar 2019 10:55:45 +0000 (11:55 +0100)]
Dashboard: only show worker list if there are workers

This prevents an empty table with a useless action bar from being shown.

5 weeks agoJavaScript improvements to error handling & reporting in JWT flow
Sybren A. Stüvel [Tue, 19 Mar 2019 10:32:37 +0000 (11:32 +0100)]
JavaScript improvements to error handling & reporting in JWT flow

5 weeks agoMade JWT token authentication optional
Sybren A. Stüvel [Tue, 19 Mar 2019 09:20:05 +0000 (10:20 +0100)]
Made JWT token authentication optional

It is enabled by default, and produces a warning when disabled.

5 weeks agoProtect the dashboard with JWT tokens
Sybren A. Stüvel [Mon, 18 Mar 2019 13:17:24 +0000 (14:17 +0100)]
Protect the dashboard with JWT tokens

Upon receiving a 401 response from `/as-json`, the JS code will now GET
the `/jwt/token-urls` endpoint, which returns URLs to call to A) get the
token, and B) log in if URL A) returns 403 Forbidden.

Protecting the latest image meant that we no longer only protect XHR
calls, but also regular HTTP requests from the browser. This was easiest
to do by moving the token from the `Authorization: Bearer {token}`
header to a cookie.

I also changed endpoint /jwt/get-token to /jwt/token-urls. The endpoint
was intended to redirect to a Flamenco Server URL where the token could
be obtained (hence the name), but that didn't work well with JavaScript
when the user also needs to log in on Flamenco Server. Just returning
JSON with the two URLs ('get token' and 'login') turned out to be the
simplest approach.

5 weeks agoSeparated JWT auth from Shaman
Sybren A. Stüvel [Mon, 18 Mar 2019 10:21:34 +0000 (11:21 +0100)]
Separated JWT auth from Shaman

Previously the JWT auth stuff was only used for the Shaman component, but
now that we want to use it to secure access to the Dashboard as well, we
have to take it out and make it completely separate.

5 weeks agoAdded endpoint for getting a JWT token with JavaScript
Sybren A. Stüvel [Fri, 15 Mar 2019 16:54:18 +0000 (17:54 +0100)]
Added endpoint for getting a JWT token with JavaScript

This endpoint actually redirects to Flamenco Server (with expiry date +
HMAC in that link so the Server can verify it's us) to its token generation

5 weeks agoAdded MIT license text to JavaScript files
Sybren A. Stüvel [Mon, 18 Mar 2019 10:30:07 +0000 (11:30 +0100)]
Added MIT license text to JavaScript files

5 weeks agoFix double-close of a image watcher channel
Sybren A. Stüvel [Mon, 18 Mar 2019 16:27:11 +0000 (17:27 +0100)]
Fix double-close of a image watcher channel

5 weeks agoFix: test task creation created `{render}` directory
Sybren A. Stüvel [Mon, 18 Mar 2019 09:49:58 +0000 (10:49 +0100)]
Fix: test task creation created `{render}` directory

No variable replacement was done before creating the render output
directory, which is now fixed.

5 weeks agoGetTestConfig() now explicitly loads from the flamenco source directory
Sybren A. Stüvel [Mon, 18 Mar 2019 09:49:14 +0000 (10:49 +0100)]
GetTestConfig() now explicitly loads from the flamenco source directory

This makes test runs not depend on the current working directory any more,
making it easier to use from other packages and improving stability.

5 weeks agoTemplatePathPrefix now also searches in the current source directory
Sybren A. Stüvel [Mon, 18 Mar 2019 09:48:28 +0000 (10:48 +0100)]
TemplatePathPrefix now also searches in the current source directory

This makes usage in unit tests not depend on the current working directory
any more, making them more reliable.

5 weeks agoRenamed some functions in testtask.go
Sybren A. Stüvel [Mon, 18 Mar 2019 09:47:42 +0000 (10:47 +0100)]
Renamed some functions in testtask.go

Those functions were mentioning 'test jobs', which we never had, and also
mentioned sending those to the Server, which we never did.

5 weeks agoRenamed testjob.go to testtask.go
Sybren A. Stüvel [Mon, 18 Mar 2019 08:44:41 +0000 (09:44 +0100)]
Renamed testjob.go to testtask.go

5 weeks agoMerge branch 'hotfix-v2.3'
Sybren A. Stüvel [Mon, 18 Mar 2019 08:41:42 +0000 (09:41 +0100)]
Merge branch 'hotfix-v2.3'

5 weeks agoBumped version to 2.3.1 hotfix-v2.3 v2.3.1
Sybren A. Stüvel [Fri, 15 Mar 2019 17:06:54 +0000 (18:06 +0100)]
Bumped version to 2.3.1

5 weeks agoFix panic when handling jobless tasks
Sybren A. Stüvel [Fri, 15 Mar 2019 17:05:54 +0000 (18:05 +0100)]
Fix panic when handling jobless tasks

Previously test tasks were created without job ID, which interfered with
storing the task logs on the Manager (because the job ID is used there for
the path). We now create test tasks with job ID 000000000000000000000000.

5 weeks agoMoved Flamenco Manager HTTP endpoint code into a separate file
Sybren A. Stüvel [Fri, 15 Mar 2019 15:18:35 +0000 (16:18 +0100)]
Moved Flamenco Manager HTTP endpoint code into a separate file

Probably it'll be better to properly refactor the flamenco package and
move the routes in there, but for now this at least cleans up the main.go
a bit.

5 weeks agoGarbage Collect: Perform GC sweep after the configured period
Sybren A. Stüvel [Fri, 15 Mar 2019 14:51:51 +0000 (15:51 +0100)]
Garbage Collect: Perform GC sweep after the configured period

This makes the GC sweep only after X hours (depending on configuration)
instead of 3 seconds after the Flamenco Manager starts up. The new
behaviour makes it less stressful on the system to frequently redeploy new
versions of Flamenco Manager.

5 weeks agoStart JWT download loop only if shamanserver.Go() is called
Sybren A. Stüvel [Fri, 15 Mar 2019 14:49:36 +0000 (15:49 +0100)]
Start JWT download loop only if shamanserver.Go() is called

This makes the background stuff start explicitly again. The lifecycle of
the JWT download loop is still tied to the Shaman server, even though I
moved the `jwtauth` package out of it (88d48a405b58856dc15b595f5dcf3b01528630a6).

5 weeks agoSilently disable periodic garbage collection sweep when using `-gc` CLI arg
Sybren A. Stüvel [Fri, 15 Mar 2019 14:38:49 +0000 (15:38 +0100)]
Silently disable periodic garbage collection sweep when using `-gc` CLI arg

5 weeks agoCLI args for performing a garbage collection sweep
Sybren A. Stüvel [Fri, 15 Mar 2019 14:31:43 +0000 (15:31 +0100)]
CLI args for performing a garbage collection sweep

Use `-gc` to do a dry-run of the garbage collection.
Use `-gc -i-know-what-i-am-doing` to actually perform the GC and delete

5 weeks agoGarbage Collect: Reduce logging when checking symlinks
Sybren A. Stüvel [Fri, 15 Mar 2019 14:27:55 +0000 (15:27 +0100)]
Garbage Collect: Reduce logging when checking symlinks

Hopefully this makes a GC sweep a bit faster.

5 weeks agoGarbage Collect: collect & log more statistics
Sybren A. Stüvel [Fri, 15 Mar 2019 14:27:31 +0000 (15:27 +0100)]
Garbage Collect: collect & log more statistics

We now also log the number of symlinks inspected, and this addition was
enough to refactor the collection of statistics into its own struct.

5 weeks agoLog shaman as package 'shaman' and not the old 'shaman/libshaman'
Sybren A. Stüvel [Fri, 15 Mar 2019 14:24:57 +0000 (15:24 +0100)]
Log shaman as package 'shaman' and not the old 'shaman/libshaman'

5 weeks agoMoved …/shaman/auth to …/jwauth
Sybren A. Stüvel [Fri, 15 Mar 2019 11:43:58 +0000 (12:43 +0100)]
Moved …/shaman/auth to …/jwauth

5 weeks agoRemoved Shaman packages from Gopkg.lock
Sybren A. Stüvel [Fri, 15 Mar 2019 11:44:16 +0000 (12:44 +0100)]
Removed Shaman packages from Gopkg.lock

6 weeks agoRemoved some test endpoints
Sybren A. Stüvel [Wed, 13 Mar 2019 13:41:25 +0000 (14:41 +0100)]
Removed some test endpoints

6 weeks agoAdded test route for debugging JWT tokens
Sybren A. Stüvel [Wed, 13 Mar 2019 12:41:49 +0000 (13:41 +0100)]
Added test route for debugging JWT tokens

6 weeks agoAuto-download JWT public keys
Sybren A. Stüvel [Fri, 8 Mar 2019 17:03:24 +0000 (18:03 +0100)]
Auto-download JWT public keys

The public keys are downloaded in a loop with a configurable interval
(default 4h). The HTTP `Last-Modified` and `If-Modified-Since` headers
are used to minimise download overhead and to only reload the keys when
they have actually changed.

After startup and after downloading new keys the Shaman performs a
self-test of the JWT tokens. If it has a private key, it tests that it
has a matching public key; if not, it logs an error.

6 weeks agoMake all tests (not just short ones)
Sybren A. Stüvel [Wed, 13 Mar 2019 10:48:51 +0000 (11:48 +0100)]
Make all tests (not just short ones)

6 weeks agoMade unit tests faster by connecting to MongoDB only on suite start
Sybren A. Stüvel [Wed, 13 Mar 2019 11:50:52 +0000 (12:50 +0100)]
Made unit tests faster by connecting to MongoDB only on suite start

Instead of connecting for every test, we now connect for every test suite
only. This reduces the runtime of the tests from 23s to 15s on my machine.

6 weeks agoDon't overwrite latest-image.jpg when deploying
Sybren A. Stüvel [Tue, 12 Mar 2019 16:55:22 +0000 (17:55 +0100)]
Don't overwrite latest-image.jpg when deploying

6 weeks agoPass correct worker ID when un-blacklisting
Sybren A. Stüvel [Tue, 12 Mar 2019 16:52:39 +0000 (17:52 +0100)]
Pass correct worker ID when un-blacklisting

6 weeks agoReordered some imports
Sybren A. Stüvel [Tue, 12 Mar 2019 15:54:25 +0000 (16:54 +0100)]
Reordered some imports

6 weeks agoRelicensed from GPLv3 to MIT license
Sybren A. Stüvel [Tue, 12 Mar 2019 15:54:00 +0000 (16:54 +0100)]
Relicensed from GPLv3 to MIT license

6 weeks agoSimplify version nr when exactly on a tag
Sybren A. Stüvel [Tue, 12 Mar 2019 15:14:45 +0000 (16:14 +0100)]
Simplify version nr when exactly on a tag

Instead of marking as `vX.Y-0-hashash` it's now just using `vX.Y`.

6 weeks agoRemoved deploy.sh; deploy with `make deploy` instead
Sybren A. Stüvel [Tue, 12 Mar 2019 14:32:35 +0000 (15:32 +0100)]
Removed deploy.sh; deploy with `make deploy` instead

6 weeks agoUpdated CHANGELOG
Sybren A. Stüvel [Tue, 12 Mar 2019 14:28:57 +0000 (15:28 +0100)]

6 weeks agoAdded JWT keys used in unit tests
Sybren A. Stüvel [Fri, 8 Mar 2019 16:26:02 +0000 (17:26 +0100)]
Added JWT keys used in unit tests

6 weeks agoDocumented Shaman endpoints
Sybren A. Stüvel [Fri, 8 Mar 2019 16:25:02 +0000 (17:25 +0100)]
Documented Shaman endpoints

6 weeks agoRemoved Shaman http server
Sybren A. Stüvel [Fri, 8 Mar 2019 16:11:53 +0000 (17:11 +0100)]
Removed Shaman http server

6 weeks agoMoved Shaman code into Flamenco Manager
Sybren A. Stüvel [Fri, 8 Mar 2019 16:04:49 +0000 (17:04 +0100)]
Moved Shaman code into Flamenco Manager

This Shaman code is a copy of revision 195ee4b02c715b9ec1936aa226f2cb10ea7fe32a
at https://gitlab.com/blender-institute/shaman. That repository will most
likely not see any more updates, and Shaman will become a part of Flamenco