blender-my-data.git
13 months agoAdded cron job to deployment documentation master
Sybren A. Stüvel [Fri, 24 May 2019 08:40:21 +0000 (10:40 +0200)]
Added cron job to deployment documentation

18 months agoCross-platform support for sed command
Francesco Siddi [Tue, 11 Dec 2018 17:14:06 +0000 (18:14 +0100)]
Cross-platform support for sed command

18 months agoUpdate LATEST_SCHEMA_VERSION to 2 production
Francesco Siddi [Tue, 11 Dec 2018 14:56:00 +0000 (15:56 +0100)]
Update LATEST_SCHEMA_VERSION to 2

18 months agoDocs: Make port number consistent
Francesco Siddi [Tue, 11 Dec 2018 14:53:28 +0000 (15:53 +0100)]
Docs: Make port number consistent

18 months agoRename example.sh to submit_benchmark.sh
Francesco Siddi [Tue, 11 Dec 2018 14:53:00 +0000 (15:53 +0100)]
Rename example.sh to submit_benchmark.sh

19 months agoUse TCP/IP sockets instead of UNIX sockets for uWSGI
Sybren A. Stüvel [Wed, 7 Nov 2018 11:49:50 +0000 (12:49 +0100)]
Use TCP/IP sockets instead of UNIX sockets for uWSGI

The UNIX sockets sometimes timed out, so this is a test to see if TCP/IP
sockets work better.

19 months agoFixed deleting not-yet-synced benchmarks
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:43 +0000 (11:32 +0100)]
Fixed deleting not-yet-synced benchmarks

19 months agoLogging improvements
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:27 +0000 (11:32 +0100)]
Logging improvements

19 months agoFormatting
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:10 +0000 (11:32 +0100)]
Formatting

19 months agoReduced timeout between NginX and uWSGI + lowered uWSGI max-requests
Sybren A. Stüvel [Wed, 7 Nov 2018 10:06:38 +0000 (11:06 +0100)]
Reduced timeout between NginX and uWSGI + lowered uWSGI max-requests

This is an attempt to figure out/fix the timeouts between My Data and
Open Data.

19 months agoSome more useful commands in bash_history
Sybren A. Stüvel [Wed, 7 Nov 2018 10:06:09 +0000 (11:06 +0100)]
Some more useful commands in bash_history

19 months agoUpgraded blender-id-oauth-client 0.7 → 0.8
Sybren A. Stüvel [Wed, 7 Nov 2018 09:49:09 +0000 (10:49 +0100)]
Upgraded blender-id-oauth-client 0.7 → 0.8

19 months agoAdded example crontab
Sybren A. Stüvel [Tue, 6 Nov 2018 17:30:57 +0000 (18:30 +0100)]
Added example crontab

19 months agomark manage-{my,open}data.sh as executable
Sybren A. Stüvel [Tue, 6 Nov 2018 17:24:55 +0000 (18:24 +0100)]
mark manage-{my,open}data.sh as executable

19 months agoLinux fix
Sybren A. Stüvel [Tue, 6 Nov 2018 16:44:28 +0000 (17:44 +0100)]
Linux fix

19 months agodocker/deploy → docker/staging
Sybren A. Stüvel [Tue, 6 Nov 2018 16:44:11 +0000 (17:44 +0100)]
docker/deploy → docker/staging

For clarity, and to be in sync with what we do in the Blender Dev Fund
website.

19 months agoReworked Docker image into two parts
Sybren A. Stüvel [Tue, 6 Nov 2018 16:40:42 +0000 (17:40 +0100)]
Reworked Docker image into two parts

This is the same approach as used in the Blender Dev Fund site. The
heavy/time-consuming installation of dependencies is done in the base image,
and the project files are contained in the final image.

19 months agoCreate database index on Benchmark.date_created
Sybren A. Stüvel [Tue, 6 Nov 2018 16:30:31 +0000 (17:30 +0100)]
Create database index on Benchmark.date_created

We use this field as default sort field, so an index should speed things up
in the admin.

19 months agoAdd missing migration
Sybren A. Stüvel [Tue, 6 Nov 2018 16:25:36 +0000 (17:25 +0100)]
Add missing migration

19 months agoRemove trailing spaces
Sybren A. Stüvel [Tue, 6 Nov 2018 16:13:51 +0000 (17:13 +0100)]
Remove trailing spaces

19 months agoAdmin tweaks for benchmarks
Sybren A. Stüvel [Tue, 6 Nov 2018 16:12:31 +0000 (17:12 +0100)]
Admin tweaks for benchmarks

19 months agoQueue submissions to Open Data when it's unreachable
Sybren A. Stüvel [Tue, 6 Nov 2018 16:12:20 +0000 (17:12 +0100)]
Queue submissions to Open Data when it's unreachable

19 months agoMigrate from custom oauth module to blender-id-oauth-client
Sybren A. Stüvel [Tue, 6 Nov 2018 10:51:43 +0000 (11:51 +0100)]
Migrate from custom oauth module to blender-id-oauth-client

The blender-id-oauth-client project was actually born from the My Data
oauth module, so the database layout is pretty similar. However, since
migration with the Django migration system is hard to get right (because
of ordering issues, we'd need the blender-id-oauth-client migrations to
be dependent on MyData migrations, which would break compatibility with any
other project), some manual intervention is necessary; see the README for
this.

19 months agoMade unittests less dependent on local configuration
Sybren A. Stüvel [Tue, 6 Nov 2018 10:51:56 +0000 (11:51 +0100)]
Made unittests less dependent on local configuration

20 months agoTweaks to prep_elastic.sh
Francesco Siddi [Sun, 21 Oct 2018 11:51:36 +0000 (13:51 +0200)]
Tweaks to prep_elastic.sh

Ensure creation of deploy directory and make sed -i work on macOS.

20 months agoDeleted one line too many
Sybren A. Stüvel [Tue, 9 Oct 2018 16:17:13 +0000 (18:17 +0200)]
Deleted one line too many

20 months agoNo longer copy docker-compose.yml to server
Sybren A. Stüvel [Tue, 9 Oct 2018 15:38:05 +0000 (17:38 +0200)]
No longer copy docker-compose.yml to server

It's now managed independently of this project, as it is a cross-project
file.

20 months agoUpdated REMOTE_DOCKER_COMPOSE_DIR
Sybren A. Stüvel [Tue, 9 Oct 2018 15:37:25 +0000 (17:37 +0200)]
Updated REMOTE_DOCKER_COMPOSE_DIR

20 months agoRemove TLS from docker image in favour of letting Træfik handle it
Sybren A. Stüvel [Tue, 9 Oct 2018 14:29:24 +0000 (16:29 +0200)]
Remove TLS from docker image in favour of letting Træfik handle it

20 months agoLinux fix
Sybren A. Stüvel [Tue, 9 Oct 2018 14:37:40 +0000 (16:37 +0200)]
Linux fix

21 months agoAddress concerns raised in rBMD74e3f65e8fbf
Francesco Siddi [Sun, 9 Sep 2018 16:20:38 +0000 (18:20 +0200)]
Address concerns raised in rBMD74e3f65e8fbf

22 months agoCleanly shut down uwsgi when stopping the docker container + nginx last
Sybren A. Stüvel [Wed, 29 Aug 2018 07:20:50 +0000 (09:20 +0200)]
Cleanly shut down uwsgi when stopping the docker container + nginx last

Stopping nginx last should keep the friendly '502 bad gateway' page
available for as long as possible before shutting down completely.

Stopping uwsgi cleanly should prevent OSErrors trying to write somewhere
while docker is being shut down.

22 months agodeploy.sh: check whether Docker is alive before doing anything else
Sybren A. Stüvel [Wed, 29 Aug 2018 07:10:52 +0000 (09:10 +0200)]
deploy.sh: check whether Docker is alive before doing anything else

22 months agoForward REMOTE_ADDR as X-Forwarded-For header to Open Data
Sybren A. Stüvel [Wed, 29 Aug 2018 06:54:46 +0000 (08:54 +0200)]
Forward REMOTE_ADDR as X-Forwarded-For header to Open Data

This will allow us to see the original address for the original request
in the Open Data Sentry logs.

22 months agoAdded note about why we check the client version way before using it
Sybren A. Stüvel [Wed, 29 Aug 2018 06:53:50 +0000 (08:53 +0200)]
Added note about why we check the client version way before using it

22 months agoUse Requests session to submit to opendata
Sybren A. Stüvel [Wed, 29 Aug 2018 06:53:35 +0000 (08:53 +0200)]
Use Requests session to submit to opendata

This ensures that we can reuse the HTTPS connection and don't need TCP/IP
and TLS handshakes for every single submission. It also allows auto-
retrying if there are any hickups.

22 months agoUpdated example.sh to work on current mydata
Sybren A. Stüvel [Tue, 28 Aug 2018 13:31:04 +0000 (15:31 +0200)]
Updated example.sh to work on current mydata

22 months agoAdded Raven to Pipefile for Sentry.io integration
Sybren A. Stüvel [Tue, 28 Aug 2018 11:07:03 +0000 (13:07 +0200)]
Added Raven to Pipefile for Sentry.io integration

22 months agoUse pathlib and BASE_DIR in example Raven configuration
Sybren A. Stüvel [Tue, 28 Aug 2018 11:04:31 +0000 (13:04 +0200)]
Use pathlib and BASE_DIR in example Raven configuration

22 months agoSupport multiple schema versions; introduces Benchmark schema v2
Sybren A. Stüvel [Tue, 28 Aug 2018 10:44:25 +0000 (12:44 +0200)]
Support multiple schema versions; introduces Benchmark schema v2

Since the validation and possible schema upgrades are done at the Open
Data side, there isn't much to change here.

22 months agoUse Django asserts
Francesco Siddi [Sun, 26 Aug 2018 21:55:40 +0000 (23:55 +0200)]
Use Django asserts

22 months agoExtend tests for benchmark deletion
Francesco Siddi [Sun, 26 Aug 2018 21:55:16 +0000 (23:55 +0200)]
Extend tests for benchmark deletion

22 months agoExtend description of ConnectionError handling message
Francesco Siddi [Sun, 26 Aug 2018 21:50:02 +0000 (23:50 +0200)]
Extend description of ConnectionError handling message

22 months agoDisplay samples not deleted only if they exist
Francesco Siddi [Sun, 26 Aug 2018 21:49:26 +0000 (23:49 +0200)]
Display samples not deleted only if they exist

22 months agoAdd utility methods to AbstractBenchmarkTest
Francesco Siddi [Sun, 26 Aug 2018 21:43:28 +0000 (23:43 +0200)]
Add utility methods to AbstractBenchmarkTest

Methods to bulk add benchmarks and to mock the
deletion response from Open Data.

22 months agoAdd test docstring
Francesco Siddi [Sat, 25 Aug 2018 19:57:09 +0000 (21:57 +0200)]
Add test docstring

22 months agoExtend test_benchmark_deletion
Francesco Siddi [Sat, 25 Aug 2018 19:53:37 +0000 (21:53 +0200)]
Extend test_benchmark_deletion

22 months agoWrap lines at 100 chars
Francesco Siddi [Sat, 25 Aug 2018 18:30:53 +0000 (20:30 +0200)]
Wrap lines at 100 chars

22 months agoImprove samples deletion
Francesco Siddi [Sat, 25 Aug 2018 09:40:22 +0000 (11:40 +0200)]
Improve samples deletion

22 months agoImprove javascript utilities
Francesco Siddi [Sat, 25 Aug 2018 09:39:56 +0000 (11:39 +0200)]
Improve javascript utilities

Move xhr related functions in dedicated js files.

22 months agoTest for benchmark deletion with ConnectionError
Francesco Siddi [Fri, 24 Aug 2018 13:13:00 +0000 (15:13 +0200)]
Test for benchmark deletion with ConnectionError

22 months agoTest deletion of a non-existing benchmark
Francesco Siddi [Thu, 23 Aug 2018 19:12:38 +0000 (21:12 +0200)]
Test deletion of a non-existing benchmark

22 months agoExtend test for single benchmark deletion
Francesco Siddi [Thu, 23 Aug 2018 18:58:54 +0000 (20:58 +0200)]
Extend test for single benchmark deletion

22 months agoExtend opendata_mock_happy_submission
Francesco Siddi [Thu, 23 Aug 2018 18:57:45 +0000 (20:57 +0200)]
Extend opendata_mock_happy_submission

By specifying manage_id and benchmark_id we can mock
multiple calls to the api, and submit multiple benchmarks.

22 months agoStarted working on unit test for sample deletion
Sybren A. Stüvel [Thu, 23 Aug 2018 15:56:00 +0000 (17:56 +0200)]
Started working on unit test for sample deletion

22 months agoAdded logging to sample deletion
Sybren A. Stüvel [Thu, 23 Aug 2018 15:55:28 +0000 (17:55 +0200)]
Added logging to sample deletion

22 months agoAdd auth header when requesting sample deletion at Open Data
Sybren A. Stüvel [Thu, 23 Aug 2018 15:55:20 +0000 (17:55 +0200)]
Add auth header when requesting sample deletion at Open Data

22 months agonginx: Allow listing of downloads directory
Sybren A. Stüvel [Thu, 23 Aug 2018 10:21:12 +0000 (12:21 +0200)]
nginx: Allow listing of downloads directory

22 months agoUpdated `elastic-dump.sh` for new usage
Sybren A. Stüvel [Thu, 23 Aug 2018 09:54:32 +0000 (11:54 +0200)]
Updated `elastic-dump.sh` for new usage

See Open Data, commit 1f4a0f0ce3faf147f622f05be1432bc38a570fbe

22 months agoDocker entrypoint: doing fail when {open,my}data.log doesn't exist
Sybren A. Stüvel [Thu, 23 Aug 2018 08:17:49 +0000 (10:17 +0200)]
Docker entrypoint: doing fail when {open,my}data.log doesn't exist

22 months agoelastic-dump.sh should be executable
Sybren A. Stüvel [Thu, 23 Aug 2018 08:17:35 +0000 (10:17 +0200)]
elastic-dump.sh should be executable

22 months agoAdded script to easily perform the 'Elastic Dump' in Docker
Sybren A. Stüvel [Wed, 22 Aug 2018 15:00:17 +0000 (17:00 +0200)]
Added script to easily perform the 'Elastic Dump' in Docker

22 months agoMoved files that go into the root of the docker image into their own dir
Sybren A. Stüvel [Wed, 22 Aug 2018 14:59:51 +0000 (16:59 +0200)]
Moved files that go into the root of the docker image into their own dir

22 months agoOpenData can now return a 401 or 403 depending on the exact situation
Sybren A. Stüvel [Wed, 22 Aug 2018 13:59:16 +0000 (15:59 +0200)]
OpenData can now return a 401 or 403 depending on the exact situation

22 months agoEnsure Python log files are writable before start uWSGI
Sybren A. Stüvel [Tue, 21 Aug 2018 15:42:02 +0000 (17:42 +0200)]
Ensure Python log files are writable before start uWSGI

22 months agoUse fonts from Blender Web Assets
Pablo Vazquez [Fri, 17 Aug 2018 15:57:05 +0000 (17:57 +0200)]
Use fonts from Blender Web Assets

22 months agoDelete revoked ClientTokens
Francesco Siddi [Fri, 17 Aug 2018 15:37:51 +0000 (17:37 +0200)]
Delete revoked ClientTokens

Since we are not using the is_valid field in ClientToken,
we remove it.

22 months agoImprove text in Token section
Francesco Siddi [Fri, 17 Aug 2018 14:32:06 +0000 (16:32 +0200)]
Improve text in Token section

22 months agoFormatting
Sybren A. Stüvel [Fri, 17 Aug 2018 08:54:36 +0000 (10:54 +0200)]
Formatting

22 months agoUse ClientSettings.url_xxxx for all Blender ID URLs
Sybren A. Stüvel [Fri, 17 Aug 2018 08:54:24 +0000 (10:54 +0200)]
Use ClientSettings.url_xxxx for all Blender ID URLs

22 months agoMade test actually test the right thing
Sybren A. Stüvel [Fri, 17 Aug 2018 08:53:51 +0000 (10:53 +0200)]
Made test actually test the right thing

My previous 'fix' did the wrong thing.

22 months agoFixed failing unit tests
Sybren A. Stüvel [Fri, 17 Aug 2018 08:37:28 +0000 (10:37 +0200)]
Fixed failing unit tests

One should always run the entire unit test suite, especially when using
lru_cache, as that makes tests interfere with each other.

22 months agoAdded mypy testing
Sybren A. Stüvel [Fri, 17 Aug 2018 08:29:25 +0000 (10:29 +0200)]
Added mypy testing

22 months agoAdd TODO for brighter days
Francesco Siddi [Thu, 16 Aug 2018 17:12:10 +0000 (19:12 +0200)]
Add TODO for brighter days

22 months agoAdd token revoke workflow
Francesco Siddi [Thu, 16 Aug 2018 17:09:35 +0000 (19:09 +0200)]
Add token revoke workflow

Introduce a new /token/revoke end point that accepts
POST requests with a token_id. If the request is
valid, the token is updated to is_valid=False.

Fixes T56380
Fixes T56399

22 months agoMove Blender ID OAuth client into named tuple
Francesco Siddi [Thu, 16 Aug 2018 13:47:51 +0000 (15:47 +0200)]
Move Blender ID OAuth client into named tuple

Does not crash the app when BLENDER_ID is
not configured!

22 months agoInclude the Django Admin docs
Sybren A. Stüvel [Thu, 16 Aug 2018 10:30:11 +0000 (12:30 +0200)]
Include the Django Admin docs

22 months agoDo version checking of the Benchmark Client
Sybren A. Stüvel [Thu, 16 Aug 2018 10:29:41 +0000 (12:29 +0200)]
Do version checking of the Benchmark Client

- The result JSON now includes the benchmark client version, and as a
  result the schema version has been bumped to 1. This version is not
  checked yet on the server, so it's at the moment purely cosmetic.
- Latest version of the Benchmark Client is configured in
  LATEST_CLIENT_VERSIONS['BENCHMARK']. This allows us to have different
  clients with different latest versions.
- An empty string as 'latest version' disables the version check altogether.
  This allows us to push to the production servers before we release the
  client version that actually includes the version information.

22 months agoShow timezone in 'Your Benchmarks'
Sybren A. Stüvel [Thu, 16 Aug 2018 08:48:10 +0000 (10:48 +0200)]
Show timezone in 'Your Benchmarks'

Showing time without timezone is meaningless.

22 months agoReformatted example JSON
Sybren A. Stüvel [Thu, 16 Aug 2018 08:47:45 +0000 (10:47 +0200)]
Reformatted example JSON

We use four spaces for JSON in all our projects.

22 months agoSplit views.py into separate files
Sybren A. Stüvel [Wed, 15 Aug 2018 14:21:03 +0000 (16:21 +0200)]
Split views.py into separate files

22 months agoBasic admin backend customization
Francesco Siddi [Wed, 15 Aug 2018 14:53:49 +0000 (16:53 +0200)]
Basic admin backend customization

Display site name in admin header templates.
Fixes T56400

22 months agoMatch Blender ID navigation
Francesco Siddi [Wed, 15 Aug 2018 13:30:33 +0000 (15:30 +0200)]
Match Blender ID navigation

22 months agoFix T56381: store hostname with client auth token
Sybren A. Stüvel [Wed, 15 Aug 2018 11:07:40 +0000 (13:07 +0200)]
Fix T56381: store hostname with client auth token

22 months agoChange default ordering of Benchmark and ClientToken to newest-first
Sybren A. Stüvel [Wed, 15 Aug 2018 11:07:14 +0000 (13:07 +0200)]
Change default ordering of Benchmark and ClientToken to newest-first

22 months agoUpdated webassets
Sybren A. Stüvel [Wed, 15 Aug 2018 10:53:18 +0000 (12:53 +0200)]
Updated webassets

22 months agoDouble quotes for consistency
Francesco Siddi [Wed, 15 Aug 2018 10:30:59 +0000 (12:30 +0200)]
Double quotes for consistency

22 months agoAdd favicon
Francesco Siddi [Wed, 15 Aug 2018 10:28:03 +0000 (12:28 +0200)]
Add favicon

22 months agoUpdate nginx configuration
Francesco Siddi [Wed, 15 Aug 2018 10:04:19 +0000 (12:04 +0200)]
Update nginx configuration

Now static files for opendata are located in /static/

22 months agoRemove unused config file
Francesco Siddi [Wed, 15 Aug 2018 09:13:30 +0000 (11:13 +0200)]
Remove unused config file

22 months agoConform directory layout
Francesco Siddi [Wed, 15 Aug 2018 09:13:17 +0000 (11:13 +0200)]
Conform directory layout

Move static to webstatic and src to websrc.
This way when we run ./manage.py collectstatic,
files will be copied to the standard static location.

Fixes T56402

22 months agoUpdate repo links to new official location
Francesco Siddi [Tue, 14 Aug 2018 14:27:16 +0000 (16:27 +0200)]
Update repo links to new official location

22 months agoAdd terms of service and privacy policy links
Pablo Vazquez [Tue, 14 Aug 2018 14:10:54 +0000 (16:10 +0200)]
Add terms of service and privacy policy links

22 months agoPadding mismatch with Blender ID
Pablo Vazquez [Tue, 14 Aug 2018 14:06:58 +0000 (16:06 +0200)]
Padding mismatch with Blender ID

22 months agoTweak prep_elastic.sh to run on macOS
Francesco Siddi [Fri, 10 Aug 2018 19:03:13 +0000 (21:03 +0200)]
Tweak prep_elastic.sh to run on macOS

22 months agoShow a nice message for a 502 Bad Gateway error
Sybren A. Stüvel [Fri, 10 Aug 2018 15:14:59 +0000 (17:14 +0200)]
Show a nice message for a 502 Bad Gateway error

22 months agoAuthentication -> Applications
Pablo Vazquez [Fri, 10 Aug 2018 15:03:56 +0000 (17:03 +0200)]
Authentication -> Applications

22 months agoAdded single do-everything deploy script
Sybren A. Stüvel [Fri, 10 Aug 2018 14:51:45 +0000 (16:51 +0200)]
Added single do-everything deploy script

22 months agoProper redirect for opendata http → https
Sybren A. Stüvel [Fri, 10 Aug 2018 14:38:58 +0000 (16:38 +0200)]
Proper redirect for opendata http → https

22 months agoMyData is in beta!
Pablo Vazquez [Fri, 10 Aug 2018 14:36:15 +0000 (16:36 +0200)]
MyData is in beta!