blender-my-data.git
2 years agoAdded cron job to deployment documentation master
Sybren A. Stüvel [Fri, 24 May 2019 08:40:21 +0000 (10:40 +0200)]
Added cron job to deployment documentation

2 years agoCross-platform support for sed command
Francesco Siddi [Tue, 11 Dec 2018 17:14:06 +0000 (18:14 +0100)]
Cross-platform support for sed command

2 years agoUpdate LATEST_SCHEMA_VERSION to 2 production
Francesco Siddi [Tue, 11 Dec 2018 14:56:00 +0000 (15:56 +0100)]
Update LATEST_SCHEMA_VERSION to 2

2 years agoDocs: Make port number consistent
Francesco Siddi [Tue, 11 Dec 2018 14:53:28 +0000 (15:53 +0100)]
Docs: Make port number consistent

2 years agoRename example.sh to submit_benchmark.sh
Francesco Siddi [Tue, 11 Dec 2018 14:53:00 +0000 (15:53 +0100)]
Rename example.sh to submit_benchmark.sh

2 years agoUse TCP/IP sockets instead of UNIX sockets for uWSGI
Sybren A. Stüvel [Wed, 7 Nov 2018 11:49:50 +0000 (12:49 +0100)]
Use TCP/IP sockets instead of UNIX sockets for uWSGI

The UNIX sockets sometimes timed out, so this is a test to see if TCP/IP
sockets work better.

2 years agoFixed deleting not-yet-synced benchmarks
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:43 +0000 (11:32 +0100)]
Fixed deleting not-yet-synced benchmarks

2 years agoLogging improvements
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:27 +0000 (11:32 +0100)]
Logging improvements

2 years agoFormatting
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:10 +0000 (11:32 +0100)]
Formatting

2 years agoReduced timeout between NginX and uWSGI + lowered uWSGI max-requests
Sybren A. Stüvel [Wed, 7 Nov 2018 10:06:38 +0000 (11:06 +0100)]
Reduced timeout between NginX and uWSGI + lowered uWSGI max-requests

This is an attempt to figure out/fix the timeouts between My Data and
Open Data.

2 years agoSome more useful commands in bash_history
Sybren A. Stüvel [Wed, 7 Nov 2018 10:06:09 +0000 (11:06 +0100)]
Some more useful commands in bash_history

2 years agoUpgraded blender-id-oauth-client 0.7 → 0.8
Sybren A. Stüvel [Wed, 7 Nov 2018 09:49:09 +0000 (10:49 +0100)]
Upgraded blender-id-oauth-client 0.7 → 0.8

2 years agoAdded example crontab
Sybren A. Stüvel [Tue, 6 Nov 2018 17:30:57 +0000 (18:30 +0100)]
Added example crontab

2 years agomark manage-{my,open}data.sh as executable
Sybren A. Stüvel [Tue, 6 Nov 2018 17:24:55 +0000 (18:24 +0100)]
mark manage-{my,open}data.sh as executable

2 years agoLinux fix
Sybren A. Stüvel [Tue, 6 Nov 2018 16:44:28 +0000 (17:44 +0100)]
Linux fix

2 years agodocker/deploy → docker/staging
Sybren A. Stüvel [Tue, 6 Nov 2018 16:44:11 +0000 (17:44 +0100)]
docker/deploy → docker/staging

For clarity, and to be in sync with what we do in the Blender Dev Fund
website.

2 years agoReworked Docker image into two parts
Sybren A. Stüvel [Tue, 6 Nov 2018 16:40:42 +0000 (17:40 +0100)]
Reworked Docker image into two parts

This is the same approach as used in the Blender Dev Fund site. The
heavy/time-consuming installation of dependencies is done in the base image,
and the project files are contained in the final image.

2 years agoCreate database index on Benchmark.date_created
Sybren A. Stüvel [Tue, 6 Nov 2018 16:30:31 +0000 (17:30 +0100)]
Create database index on Benchmark.date_created

We use this field as default sort field, so an index should speed things up
in the admin.

2 years agoAdd missing migration
Sybren A. Stüvel [Tue, 6 Nov 2018 16:25:36 +0000 (17:25 +0100)]
Add missing migration

2 years agoRemove trailing spaces
Sybren A. Stüvel [Tue, 6 Nov 2018 16:13:51 +0000 (17:13 +0100)]
Remove trailing spaces

2 years agoAdmin tweaks for benchmarks
Sybren A. Stüvel [Tue, 6 Nov 2018 16:12:31 +0000 (17:12 +0100)]
Admin tweaks for benchmarks

2 years agoQueue submissions to Open Data when it's unreachable
Sybren A. Stüvel [Tue, 6 Nov 2018 16:12:20 +0000 (17:12 +0100)]
Queue submissions to Open Data when it's unreachable

2 years agoMigrate from custom oauth module to blender-id-oauth-client
Sybren A. Stüvel [Tue, 6 Nov 2018 10:51:43 +0000 (11:51 +0100)]
Migrate from custom oauth module to blender-id-oauth-client

The blender-id-oauth-client project was actually born from the My Data
oauth module, so the database layout is pretty similar. However, since
migration with the Django migration system is hard to get right (because
of ordering issues, we'd need the blender-id-oauth-client migrations to
be dependent on MyData migrations, which would break compatibility with any
other project), some manual intervention is necessary; see the README for
this.

2 years agoMade unittests less dependent on local configuration
Sybren A. Stüvel [Tue, 6 Nov 2018 10:51:56 +0000 (11:51 +0100)]
Made unittests less dependent on local configuration

2 years agoTweaks to prep_elastic.sh
Francesco Siddi [Sun, 21 Oct 2018 11:51:36 +0000 (13:51 +0200)]
Tweaks to prep_elastic.sh

Ensure creation of deploy directory and make sed -i work on macOS.

2 years agoDeleted one line too many
Sybren A. Stüvel [Tue, 9 Oct 2018 16:17:13 +0000 (18:17 +0200)]
Deleted one line too many

2 years agoNo longer copy docker-compose.yml to server
Sybren A. Stüvel [Tue, 9 Oct 2018 15:38:05 +0000 (17:38 +0200)]
No longer copy docker-compose.yml to server

It's now managed independently of this project, as it is a cross-project
file.

2 years agoUpdated REMOTE_DOCKER_COMPOSE_DIR
Sybren A. Stüvel [Tue, 9 Oct 2018 15:37:25 +0000 (17:37 +0200)]
Updated REMOTE_DOCKER_COMPOSE_DIR

2 years agoRemove TLS from docker image in favour of letting Træfik handle it
Sybren A. Stüvel [Tue, 9 Oct 2018 14:29:24 +0000 (16:29 +0200)]
Remove TLS from docker image in favour of letting Træfik handle it

2 years agoLinux fix
Sybren A. Stüvel [Tue, 9 Oct 2018 14:37:40 +0000 (16:37 +0200)]
Linux fix

2 years agoAddress concerns raised in rBMD74e3f65e8fbf
Francesco Siddi [Sun, 9 Sep 2018 16:20:38 +0000 (18:20 +0200)]
Address concerns raised in rBMD74e3f65e8fbf

2 years agoCleanly shut down uwsgi when stopping the docker container + nginx last
Sybren A. Stüvel [Wed, 29 Aug 2018 07:20:50 +0000 (09:20 +0200)]
Cleanly shut down uwsgi when stopping the docker container + nginx last

Stopping nginx last should keep the friendly '502 bad gateway' page
available for as long as possible before shutting down completely.

Stopping uwsgi cleanly should prevent OSErrors trying to write somewhere
while docker is being shut down.

2 years agodeploy.sh: check whether Docker is alive before doing anything else
Sybren A. Stüvel [Wed, 29 Aug 2018 07:10:52 +0000 (09:10 +0200)]
deploy.sh: check whether Docker is alive before doing anything else

2 years agoForward REMOTE_ADDR as X-Forwarded-For header to Open Data
Sybren A. Stüvel [Wed, 29 Aug 2018 06:54:46 +0000 (08:54 +0200)]
Forward REMOTE_ADDR as X-Forwarded-For header to Open Data

This will allow us to see the original address for the original request
in the Open Data Sentry logs.

2 years agoAdded note about why we check the client version way before using it
Sybren A. Stüvel [Wed, 29 Aug 2018 06:53:50 +0000 (08:53 +0200)]
Added note about why we check the client version way before using it

2 years agoUse Requests session to submit to opendata
Sybren A. Stüvel [Wed, 29 Aug 2018 06:53:35 +0000 (08:53 +0200)]
Use Requests session to submit to opendata

This ensures that we can reuse the HTTPS connection and don't need TCP/IP
and TLS handshakes for every single submission. It also allows auto-
retrying if there are any hickups.

2 years agoUpdated example.sh to work on current mydata
Sybren A. Stüvel [Tue, 28 Aug 2018 13:31:04 +0000 (15:31 +0200)]
Updated example.sh to work on current mydata

2 years agoAdded Raven to Pipefile for Sentry.io integration
Sybren A. Stüvel [Tue, 28 Aug 2018 11:07:03 +0000 (13:07 +0200)]
Added Raven to Pipefile for Sentry.io integration

2 years agoUse pathlib and BASE_DIR in example Raven configuration
Sybren A. Stüvel [Tue, 28 Aug 2018 11:04:31 +0000 (13:04 +0200)]
Use pathlib and BASE_DIR in example Raven configuration

2 years agoSupport multiple schema versions; introduces Benchmark schema v2
Sybren A. Stüvel [Tue, 28 Aug 2018 10:44:25 +0000 (12:44 +0200)]
Support multiple schema versions; introduces Benchmark schema v2

Since the validation and possible schema upgrades are done at the Open
Data side, there isn't much to change here.

2 years agoUse Django asserts
Francesco Siddi [Sun, 26 Aug 2018 21:55:40 +0000 (23:55 +0200)]
Use Django asserts

2 years agoExtend tests for benchmark deletion
Francesco Siddi [Sun, 26 Aug 2018 21:55:16 +0000 (23:55 +0200)]
Extend tests for benchmark deletion

2 years agoExtend description of ConnectionError handling message
Francesco Siddi [Sun, 26 Aug 2018 21:50:02 +0000 (23:50 +0200)]
Extend description of ConnectionError handling message

2 years agoDisplay samples not deleted only if they exist
Francesco Siddi [Sun, 26 Aug 2018 21:49:26 +0000 (23:49 +0200)]
Display samples not deleted only if they exist

2 years agoAdd utility methods to AbstractBenchmarkTest
Francesco Siddi [Sun, 26 Aug 2018 21:43:28 +0000 (23:43 +0200)]
Add utility methods to AbstractBenchmarkTest

Methods to bulk add benchmarks and to mock the
deletion response from Open Data.

2 years agoAdd test docstring
Francesco Siddi [Sat, 25 Aug 2018 19:57:09 +0000 (21:57 +0200)]
Add test docstring

2 years agoExtend test_benchmark_deletion
Francesco Siddi [Sat, 25 Aug 2018 19:53:37 +0000 (21:53 +0200)]
Extend test_benchmark_deletion

2 years agoWrap lines at 100 chars
Francesco Siddi [Sat, 25 Aug 2018 18:30:53 +0000 (20:30 +0200)]
Wrap lines at 100 chars

2 years agoImprove samples deletion
Francesco Siddi [Sat, 25 Aug 2018 09:40:22 +0000 (11:40 +0200)]
Improve samples deletion

2 years agoImprove javascript utilities
Francesco Siddi [Sat, 25 Aug 2018 09:39:56 +0000 (11:39 +0200)]
Improve javascript utilities

Move xhr related functions in dedicated js files.

2 years agoTest for benchmark deletion with ConnectionError
Francesco Siddi [Fri, 24 Aug 2018 13:13:00 +0000 (15:13 +0200)]
Test for benchmark deletion with ConnectionError

2 years agoTest deletion of a non-existing benchmark
Francesco Siddi [Thu, 23 Aug 2018 19:12:38 +0000 (21:12 +0200)]
Test deletion of a non-existing benchmark

2 years agoExtend test for single benchmark deletion
Francesco Siddi [Thu, 23 Aug 2018 18:58:54 +0000 (20:58 +0200)]
Extend test for single benchmark deletion

2 years agoExtend opendata_mock_happy_submission
Francesco Siddi [Thu, 23 Aug 2018 18:57:45 +0000 (20:57 +0200)]
Extend opendata_mock_happy_submission

By specifying manage_id and benchmark_id we can mock
multiple calls to the api, and submit multiple benchmarks.

2 years agoStarted working on unit test for sample deletion
Sybren A. Stüvel [Thu, 23 Aug 2018 15:56:00 +0000 (17:56 +0200)]
Started working on unit test for sample deletion

2 years agoAdded logging to sample deletion
Sybren A. Stüvel [Thu, 23 Aug 2018 15:55:28 +0000 (17:55 +0200)]
Added logging to sample deletion

2 years agoAdd auth header when requesting sample deletion at Open Data
Sybren A. Stüvel [Thu, 23 Aug 2018 15:55:20 +0000 (17:55 +0200)]
Add auth header when requesting sample deletion at Open Data

2 years agonginx: Allow listing of downloads directory
Sybren A. Stüvel [Thu, 23 Aug 2018 10:21:12 +0000 (12:21 +0200)]
nginx: Allow listing of downloads directory

2 years agoUpdated `elastic-dump.sh` for new usage
Sybren A. Stüvel [Thu, 23 Aug 2018 09:54:32 +0000 (11:54 +0200)]
Updated `elastic-dump.sh` for new usage

See Open Data, commit 1f4a0f0ce3faf147f622f05be1432bc38a570fbe

2 years agoDocker entrypoint: doing fail when {open,my}data.log doesn't exist
Sybren A. Stüvel [Thu, 23 Aug 2018 08:17:49 +0000 (10:17 +0200)]
Docker entrypoint: doing fail when {open,my}data.log doesn't exist

2 years agoelastic-dump.sh should be executable
Sybren A. Stüvel [Thu, 23 Aug 2018 08:17:35 +0000 (10:17 +0200)]
elastic-dump.sh should be executable

2 years agoAdded script to easily perform the 'Elastic Dump' in Docker
Sybren A. Stüvel [Wed, 22 Aug 2018 15:00:17 +0000 (17:00 +0200)]
Added script to easily perform the 'Elastic Dump' in Docker

2 years agoMoved files that go into the root of the docker image into their own dir
Sybren A. Stüvel [Wed, 22 Aug 2018 14:59:51 +0000 (16:59 +0200)]
Moved files that go into the root of the docker image into their own dir

2 years agoOpenData can now return a 401 or 403 depending on the exact situation
Sybren A. Stüvel [Wed, 22 Aug 2018 13:59:16 +0000 (15:59 +0200)]
OpenData can now return a 401 or 403 depending on the exact situation

2 years agoEnsure Python log files are writable before start uWSGI
Sybren A. Stüvel [Tue, 21 Aug 2018 15:42:02 +0000 (17:42 +0200)]
Ensure Python log files are writable before start uWSGI

2 years agoUse fonts from Blender Web Assets
Pablo Vazquez [Fri, 17 Aug 2018 15:57:05 +0000 (17:57 +0200)]
Use fonts from Blender Web Assets

2 years agoDelete revoked ClientTokens
Francesco Siddi [Fri, 17 Aug 2018 15:37:51 +0000 (17:37 +0200)]
Delete revoked ClientTokens

Since we are not using the is_valid field in ClientToken,
we remove it.

2 years agoImprove text in Token section
Francesco Siddi [Fri, 17 Aug 2018 14:32:06 +0000 (16:32 +0200)]
Improve text in Token section

2 years agoFormatting
Sybren A. Stüvel [Fri, 17 Aug 2018 08:54:36 +0000 (10:54 +0200)]
Formatting

2 years agoUse ClientSettings.url_xxxx for all Blender ID URLs
Sybren A. Stüvel [Fri, 17 Aug 2018 08:54:24 +0000 (10:54 +0200)]
Use ClientSettings.url_xxxx for all Blender ID URLs

2 years agoMade test actually test the right thing
Sybren A. Stüvel [Fri, 17 Aug 2018 08:53:51 +0000 (10:53 +0200)]
Made test actually test the right thing

My previous 'fix' did the wrong thing.

2 years agoFixed failing unit tests
Sybren A. Stüvel [Fri, 17 Aug 2018 08:37:28 +0000 (10:37 +0200)]
Fixed failing unit tests

One should always run the entire unit test suite, especially when using
lru_cache, as that makes tests interfere with each other.

2 years agoAdded mypy testing
Sybren A. Stüvel [Fri, 17 Aug 2018 08:29:25 +0000 (10:29 +0200)]
Added mypy testing

2 years agoAdd TODO for brighter days
Francesco Siddi [Thu, 16 Aug 2018 17:12:10 +0000 (19:12 +0200)]
Add TODO for brighter days

2 years agoAdd token revoke workflow
Francesco Siddi [Thu, 16 Aug 2018 17:09:35 +0000 (19:09 +0200)]
Add token revoke workflow

Introduce a new /token/revoke end point that accepts
POST requests with a token_id. If the request is
valid, the token is updated to is_valid=False.

Fixes T56380
Fixes T56399

2 years agoMove Blender ID OAuth client into named tuple
Francesco Siddi [Thu, 16 Aug 2018 13:47:51 +0000 (15:47 +0200)]
Move Blender ID OAuth client into named tuple

Does not crash the app when BLENDER_ID is
not configured!

2 years agoInclude the Django Admin docs
Sybren A. Stüvel [Thu, 16 Aug 2018 10:30:11 +0000 (12:30 +0200)]
Include the Django Admin docs

2 years agoDo version checking of the Benchmark Client
Sybren A. Stüvel [Thu, 16 Aug 2018 10:29:41 +0000 (12:29 +0200)]
Do version checking of the Benchmark Client

- The result JSON now includes the benchmark client version, and as a
  result the schema version has been bumped to 1. This version is not
  checked yet on the server, so it's at the moment purely cosmetic.
- Latest version of the Benchmark Client is configured in
  LATEST_CLIENT_VERSIONS['BENCHMARK']. This allows us to have different
  clients with different latest versions.
- An empty string as 'latest version' disables the version check altogether.
  This allows us to push to the production servers before we release the
  client version that actually includes the version information.

2 years agoShow timezone in 'Your Benchmarks'
Sybren A. Stüvel [Thu, 16 Aug 2018 08:48:10 +0000 (10:48 +0200)]
Show timezone in 'Your Benchmarks'

Showing time without timezone is meaningless.

2 years agoReformatted example JSON
Sybren A. Stüvel [Thu, 16 Aug 2018 08:47:45 +0000 (10:47 +0200)]
Reformatted example JSON

We use four spaces for JSON in all our projects.

2 years agoSplit views.py into separate files
Sybren A. Stüvel [Wed, 15 Aug 2018 14:21:03 +0000 (16:21 +0200)]
Split views.py into separate files

2 years agoBasic admin backend customization
Francesco Siddi [Wed, 15 Aug 2018 14:53:49 +0000 (16:53 +0200)]
Basic admin backend customization

Display site name in admin header templates.
Fixes T56400

2 years agoMatch Blender ID navigation
Francesco Siddi [Wed, 15 Aug 2018 13:30:33 +0000 (15:30 +0200)]
Match Blender ID navigation

2 years agoFix T56381: store hostname with client auth token
Sybren A. Stüvel [Wed, 15 Aug 2018 11:07:40 +0000 (13:07 +0200)]
Fix T56381: store hostname with client auth token

2 years agoChange default ordering of Benchmark and ClientToken to newest-first
Sybren A. Stüvel [Wed, 15 Aug 2018 11:07:14 +0000 (13:07 +0200)]
Change default ordering of Benchmark and ClientToken to newest-first

2 years agoUpdated webassets
Sybren A. Stüvel [Wed, 15 Aug 2018 10:53:18 +0000 (12:53 +0200)]
Updated webassets

2 years agoDouble quotes for consistency
Francesco Siddi [Wed, 15 Aug 2018 10:30:59 +0000 (12:30 +0200)]
Double quotes for consistency

2 years agoAdd favicon
Francesco Siddi [Wed, 15 Aug 2018 10:28:03 +0000 (12:28 +0200)]
Add favicon

2 years agoUpdate nginx configuration
Francesco Siddi [Wed, 15 Aug 2018 10:04:19 +0000 (12:04 +0200)]
Update nginx configuration

Now static files for opendata are located in /static/

2 years agoRemove unused config file
Francesco Siddi [Wed, 15 Aug 2018 09:13:30 +0000 (11:13 +0200)]
Remove unused config file

2 years agoConform directory layout
Francesco Siddi [Wed, 15 Aug 2018 09:13:17 +0000 (11:13 +0200)]
Conform directory layout

Move static to webstatic and src to websrc.
This way when we run ./manage.py collectstatic,
files will be copied to the standard static location.

Fixes T56402

2 years agoUpdate repo links to new official location
Francesco Siddi [Tue, 14 Aug 2018 14:27:16 +0000 (16:27 +0200)]
Update repo links to new official location

2 years agoAdd terms of service and privacy policy links
Pablo Vazquez [Tue, 14 Aug 2018 14:10:54 +0000 (16:10 +0200)]
Add terms of service and privacy policy links

2 years agoPadding mismatch with Blender ID
Pablo Vazquez [Tue, 14 Aug 2018 14:06:58 +0000 (16:06 +0200)]
Padding mismatch with Blender ID

2 years agoTweak prep_elastic.sh to run on macOS
Francesco Siddi [Fri, 10 Aug 2018 19:03:13 +0000 (21:03 +0200)]
Tweak prep_elastic.sh to run on macOS

2 years agoShow a nice message for a 502 Bad Gateway error
Sybren A. Stüvel [Fri, 10 Aug 2018 15:14:59 +0000 (17:14 +0200)]
Show a nice message for a 502 Bad Gateway error

2 years agoAuthentication -> Applications
Pablo Vazquez [Fri, 10 Aug 2018 15:03:56 +0000 (17:03 +0200)]
Authentication -> Applications

2 years agoAdded single do-everything deploy script
Sybren A. Stüvel [Fri, 10 Aug 2018 14:51:45 +0000 (16:51 +0200)]
Added single do-everything deploy script

2 years agoProper redirect for opendata http → https
Sybren A. Stüvel [Fri, 10 Aug 2018 14:38:58 +0000 (16:38 +0200)]
Proper redirect for opendata http → https

2 years agoMyData is in beta!
Pablo Vazquez [Fri, 10 Aug 2018 14:36:15 +0000 (16:36 +0200)]
MyData is in beta!