blender-my-data.git
4 months agoCross-platform support for sed command master
Francesco Siddi [Tue, 11 Dec 2018 17:14:06 +0000 (18:14 +0100)]
Cross-platform support for sed command

4 months agoUpdate LATEST_SCHEMA_VERSION to 2 production
Francesco Siddi [Tue, 11 Dec 2018 14:56:00 +0000 (15:56 +0100)]
Update LATEST_SCHEMA_VERSION to 2

4 months agoDocs: Make port number consistent
Francesco Siddi [Tue, 11 Dec 2018 14:53:28 +0000 (15:53 +0100)]
Docs: Make port number consistent

4 months agoRename example.sh to submit_benchmark.sh
Francesco Siddi [Tue, 11 Dec 2018 14:53:00 +0000 (15:53 +0100)]
Rename example.sh to submit_benchmark.sh

5 months agoUse TCP/IP sockets instead of UNIX sockets for uWSGI
Sybren A. Stüvel [Wed, 7 Nov 2018 11:49:50 +0000 (12:49 +0100)]
Use TCP/IP sockets instead of UNIX sockets for uWSGI

The UNIX sockets sometimes timed out, so this is a test to see if TCP/IP
sockets work better.

5 months agoFixed deleting not-yet-synced benchmarks
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:43 +0000 (11:32 +0100)]
Fixed deleting not-yet-synced benchmarks

5 months agoLogging improvements
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:27 +0000 (11:32 +0100)]
Logging improvements

5 months agoFormatting
Sybren A. Stüvel [Wed, 7 Nov 2018 10:32:10 +0000 (11:32 +0100)]
Formatting

5 months agoReduced timeout between NginX and uWSGI + lowered uWSGI max-requests
Sybren A. Stüvel [Wed, 7 Nov 2018 10:06:38 +0000 (11:06 +0100)]
Reduced timeout between NginX and uWSGI + lowered uWSGI max-requests

This is an attempt to figure out/fix the timeouts between My Data and
Open Data.

5 months agoSome more useful commands in bash_history
Sybren A. Stüvel [Wed, 7 Nov 2018 10:06:09 +0000 (11:06 +0100)]
Some more useful commands in bash_history

5 months agoUpgraded blender-id-oauth-client 0.7 → 0.8
Sybren A. Stüvel [Wed, 7 Nov 2018 09:49:09 +0000 (10:49 +0100)]
Upgraded blender-id-oauth-client 0.7 → 0.8

5 months agoAdded example crontab
Sybren A. Stüvel [Tue, 6 Nov 2018 17:30:57 +0000 (18:30 +0100)]
Added example crontab

5 months agomark manage-{my,open}data.sh as executable
Sybren A. Stüvel [Tue, 6 Nov 2018 17:24:55 +0000 (18:24 +0100)]
mark manage-{my,open}data.sh as executable

5 months agoLinux fix
Sybren A. Stüvel [Tue, 6 Nov 2018 16:44:28 +0000 (17:44 +0100)]
Linux fix

5 months agodocker/deploy → docker/staging
Sybren A. Stüvel [Tue, 6 Nov 2018 16:44:11 +0000 (17:44 +0100)]
docker/deploy → docker/staging

For clarity, and to be in sync with what we do in the Blender Dev Fund
website.

5 months agoReworked Docker image into two parts
Sybren A. Stüvel [Tue, 6 Nov 2018 16:40:42 +0000 (17:40 +0100)]
Reworked Docker image into two parts

This is the same approach as used in the Blender Dev Fund site. The
heavy/time-consuming installation of dependencies is done in the base image,
and the project files are contained in the final image.

5 months agoCreate database index on Benchmark.date_created
Sybren A. Stüvel [Tue, 6 Nov 2018 16:30:31 +0000 (17:30 +0100)]
Create database index on Benchmark.date_created

We use this field as default sort field, so an index should speed things up
in the admin.

5 months agoAdd missing migration
Sybren A. Stüvel [Tue, 6 Nov 2018 16:25:36 +0000 (17:25 +0100)]
Add missing migration

5 months agoRemove trailing spaces
Sybren A. Stüvel [Tue, 6 Nov 2018 16:13:51 +0000 (17:13 +0100)]
Remove trailing spaces

5 months agoAdmin tweaks for benchmarks
Sybren A. Stüvel [Tue, 6 Nov 2018 16:12:31 +0000 (17:12 +0100)]
Admin tweaks for benchmarks

5 months agoQueue submissions to Open Data when it's unreachable
Sybren A. Stüvel [Tue, 6 Nov 2018 16:12:20 +0000 (17:12 +0100)]
Queue submissions to Open Data when it's unreachable

5 months agoMigrate from custom oauth module to blender-id-oauth-client
Sybren A. Stüvel [Tue, 6 Nov 2018 10:51:43 +0000 (11:51 +0100)]
Migrate from custom oauth module to blender-id-oauth-client

The blender-id-oauth-client project was actually born from the My Data
oauth module, so the database layout is pretty similar. However, since
migration with the Django migration system is hard to get right (because
of ordering issues, we'd need the blender-id-oauth-client migrations to
be dependent on MyData migrations, which would break compatibility with any
other project), some manual intervention is necessary; see the README for
this.

5 months agoMade unittests less dependent on local configuration
Sybren A. Stüvel [Tue, 6 Nov 2018 10:51:56 +0000 (11:51 +0100)]
Made unittests less dependent on local configuration

6 months agoTweaks to prep_elastic.sh
Francesco Siddi [Sun, 21 Oct 2018 11:51:36 +0000 (13:51 +0200)]
Tweaks to prep_elastic.sh

Ensure creation of deploy directory and make sed -i work on macOS.

6 months agoDeleted one line too many
Sybren A. Stüvel [Tue, 9 Oct 2018 16:17:13 +0000 (18:17 +0200)]
Deleted one line too many

6 months agoNo longer copy docker-compose.yml to server
Sybren A. Stüvel [Tue, 9 Oct 2018 15:38:05 +0000 (17:38 +0200)]
No longer copy docker-compose.yml to server

It's now managed independently of this project, as it is a cross-project
file.

6 months agoUpdated REMOTE_DOCKER_COMPOSE_DIR
Sybren A. Stüvel [Tue, 9 Oct 2018 15:37:25 +0000 (17:37 +0200)]
Updated REMOTE_DOCKER_COMPOSE_DIR

6 months agoRemove TLS from docker image in favour of letting Træfik handle it
Sybren A. Stüvel [Tue, 9 Oct 2018 14:29:24 +0000 (16:29 +0200)]
Remove TLS from docker image in favour of letting Træfik handle it

6 months agoLinux fix
Sybren A. Stüvel [Tue, 9 Oct 2018 14:37:40 +0000 (16:37 +0200)]
Linux fix

7 months agoAddress concerns raised in rBMD74e3f65e8fbf
Francesco Siddi [Sun, 9 Sep 2018 16:20:38 +0000 (18:20 +0200)]
Address concerns raised in rBMD74e3f65e8fbf

7 months agoCleanly shut down uwsgi when stopping the docker container + nginx last
Sybren A. Stüvel [Wed, 29 Aug 2018 07:20:50 +0000 (09:20 +0200)]
Cleanly shut down uwsgi when stopping the docker container + nginx last

Stopping nginx last should keep the friendly '502 bad gateway' page
available for as long as possible before shutting down completely.

Stopping uwsgi cleanly should prevent OSErrors trying to write somewhere
while docker is being shut down.

7 months agodeploy.sh: check whether Docker is alive before doing anything else
Sybren A. Stüvel [Wed, 29 Aug 2018 07:10:52 +0000 (09:10 +0200)]
deploy.sh: check whether Docker is alive before doing anything else

7 months agoForward REMOTE_ADDR as X-Forwarded-For header to Open Data
Sybren A. Stüvel [Wed, 29 Aug 2018 06:54:46 +0000 (08:54 +0200)]
Forward REMOTE_ADDR as X-Forwarded-For header to Open Data

This will allow us to see the original address for the original request
in the Open Data Sentry logs.

7 months agoAdded note about why we check the client version way before using it
Sybren A. Stüvel [Wed, 29 Aug 2018 06:53:50 +0000 (08:53 +0200)]
Added note about why we check the client version way before using it

7 months agoUse Requests session to submit to opendata
Sybren A. Stüvel [Wed, 29 Aug 2018 06:53:35 +0000 (08:53 +0200)]
Use Requests session to submit to opendata

This ensures that we can reuse the HTTPS connection and don't need TCP/IP
and TLS handshakes for every single submission. It also allows auto-
retrying if there are any hickups.

7 months agoUpdated example.sh to work on current mydata
Sybren A. Stüvel [Tue, 28 Aug 2018 13:31:04 +0000 (15:31 +0200)]
Updated example.sh to work on current mydata

7 months agoAdded Raven to Pipefile for Sentry.io integration
Sybren A. Stüvel [Tue, 28 Aug 2018 11:07:03 +0000 (13:07 +0200)]
Added Raven to Pipefile for Sentry.io integration

7 months agoUse pathlib and BASE_DIR in example Raven configuration
Sybren A. Stüvel [Tue, 28 Aug 2018 11:04:31 +0000 (13:04 +0200)]
Use pathlib and BASE_DIR in example Raven configuration

7 months agoSupport multiple schema versions; introduces Benchmark schema v2
Sybren A. Stüvel [Tue, 28 Aug 2018 10:44:25 +0000 (12:44 +0200)]
Support multiple schema versions; introduces Benchmark schema v2

Since the validation and possible schema upgrades are done at the Open
Data side, there isn't much to change here.

7 months agoUse Django asserts
Francesco Siddi [Sun, 26 Aug 2018 21:55:40 +0000 (23:55 +0200)]
Use Django asserts

7 months agoExtend tests for benchmark deletion
Francesco Siddi [Sun, 26 Aug 2018 21:55:16 +0000 (23:55 +0200)]
Extend tests for benchmark deletion

7 months agoExtend description of ConnectionError handling message
Francesco Siddi [Sun, 26 Aug 2018 21:50:02 +0000 (23:50 +0200)]
Extend description of ConnectionError handling message

7 months agoDisplay samples not deleted only if they exist
Francesco Siddi [Sun, 26 Aug 2018 21:49:26 +0000 (23:49 +0200)]
Display samples not deleted only if they exist

7 months agoAdd utility methods to AbstractBenchmarkTest
Francesco Siddi [Sun, 26 Aug 2018 21:43:28 +0000 (23:43 +0200)]
Add utility methods to AbstractBenchmarkTest

Methods to bulk add benchmarks and to mock the
deletion response from Open Data.

7 months agoAdd test docstring
Francesco Siddi [Sat, 25 Aug 2018 19:57:09 +0000 (21:57 +0200)]
Add test docstring

7 months agoExtend test_benchmark_deletion
Francesco Siddi [Sat, 25 Aug 2018 19:53:37 +0000 (21:53 +0200)]
Extend test_benchmark_deletion

7 months agoWrap lines at 100 chars
Francesco Siddi [Sat, 25 Aug 2018 18:30:53 +0000 (20:30 +0200)]
Wrap lines at 100 chars

7 months agoImprove samples deletion
Francesco Siddi [Sat, 25 Aug 2018 09:40:22 +0000 (11:40 +0200)]
Improve samples deletion

7 months agoImprove javascript utilities
Francesco Siddi [Sat, 25 Aug 2018 09:39:56 +0000 (11:39 +0200)]
Improve javascript utilities

Move xhr related functions in dedicated js files.

7 months agoTest for benchmark deletion with ConnectionError
Francesco Siddi [Fri, 24 Aug 2018 13:13:00 +0000 (15:13 +0200)]
Test for benchmark deletion with ConnectionError

7 months agoTest deletion of a non-existing benchmark
Francesco Siddi [Thu, 23 Aug 2018 19:12:38 +0000 (21:12 +0200)]
Test deletion of a non-existing benchmark

8 months agoExtend test for single benchmark deletion
Francesco Siddi [Thu, 23 Aug 2018 18:58:54 +0000 (20:58 +0200)]
Extend test for single benchmark deletion

8 months agoExtend opendata_mock_happy_submission
Francesco Siddi [Thu, 23 Aug 2018 18:57:45 +0000 (20:57 +0200)]
Extend opendata_mock_happy_submission

By specifying manage_id and benchmark_id we can mock
multiple calls to the api, and submit multiple benchmarks.

8 months agoStarted working on unit test for sample deletion
Sybren A. Stüvel [Thu, 23 Aug 2018 15:56:00 +0000 (17:56 +0200)]
Started working on unit test for sample deletion

8 months agoAdded logging to sample deletion
Sybren A. Stüvel [Thu, 23 Aug 2018 15:55:28 +0000 (17:55 +0200)]
Added logging to sample deletion

8 months agoAdd auth header when requesting sample deletion at Open Data
Sybren A. Stüvel [Thu, 23 Aug 2018 15:55:20 +0000 (17:55 +0200)]
Add auth header when requesting sample deletion at Open Data

8 months agonginx: Allow listing of downloads directory
Sybren A. Stüvel [Thu, 23 Aug 2018 10:21:12 +0000 (12:21 +0200)]
nginx: Allow listing of downloads directory

8 months agoUpdated `elastic-dump.sh` for new usage
Sybren A. Stüvel [Thu, 23 Aug 2018 09:54:32 +0000 (11:54 +0200)]
Updated `elastic-dump.sh` for new usage

See Open Data, commit 1f4a0f0ce3faf147f622f05be1432bc38a570fbe

8 months agoDocker entrypoint: doing fail when {open,my}data.log doesn't exist
Sybren A. Stüvel [Thu, 23 Aug 2018 08:17:49 +0000 (10:17 +0200)]
Docker entrypoint: doing fail when {open,my}data.log doesn't exist

8 months agoelastic-dump.sh should be executable
Sybren A. Stüvel [Thu, 23 Aug 2018 08:17:35 +0000 (10:17 +0200)]
elastic-dump.sh should be executable

8 months agoAdded script to easily perform the 'Elastic Dump' in Docker
Sybren A. Stüvel [Wed, 22 Aug 2018 15:00:17 +0000 (17:00 +0200)]
Added script to easily perform the 'Elastic Dump' in Docker

8 months agoMoved files that go into the root of the docker image into their own dir
Sybren A. Stüvel [Wed, 22 Aug 2018 14:59:51 +0000 (16:59 +0200)]
Moved files that go into the root of the docker image into their own dir

8 months agoOpenData can now return a 401 or 403 depending on the exact situation
Sybren A. Stüvel [Wed, 22 Aug 2018 13:59:16 +0000 (15:59 +0200)]
OpenData can now return a 401 or 403 depending on the exact situation

8 months agoEnsure Python log files are writable before start uWSGI
Sybren A. Stüvel [Tue, 21 Aug 2018 15:42:02 +0000 (17:42 +0200)]
Ensure Python log files are writable before start uWSGI

8 months agoUse fonts from Blender Web Assets
Pablo Vazquez [Fri, 17 Aug 2018 15:57:05 +0000 (17:57 +0200)]
Use fonts from Blender Web Assets

8 months agoDelete revoked ClientTokens
Francesco Siddi [Fri, 17 Aug 2018 15:37:51 +0000 (17:37 +0200)]
Delete revoked ClientTokens

Since we are not using the is_valid field in ClientToken,
we remove it.

8 months agoImprove text in Token section
Francesco Siddi [Fri, 17 Aug 2018 14:32:06 +0000 (16:32 +0200)]
Improve text in Token section

8 months agoFormatting
Sybren A. Stüvel [Fri, 17 Aug 2018 08:54:36 +0000 (10:54 +0200)]
Formatting

8 months agoUse ClientSettings.url_xxxx for all Blender ID URLs
Sybren A. Stüvel [Fri, 17 Aug 2018 08:54:24 +0000 (10:54 +0200)]
Use ClientSettings.url_xxxx for all Blender ID URLs

8 months agoMade test actually test the right thing
Sybren A. Stüvel [Fri, 17 Aug 2018 08:53:51 +0000 (10:53 +0200)]
Made test actually test the right thing

My previous 'fix' did the wrong thing.

8 months agoFixed failing unit tests
Sybren A. Stüvel [Fri, 17 Aug 2018 08:37:28 +0000 (10:37 +0200)]
Fixed failing unit tests

One should always run the entire unit test suite, especially when using
lru_cache, as that makes tests interfere with each other.

8 months agoAdded mypy testing
Sybren A. Stüvel [Fri, 17 Aug 2018 08:29:25 +0000 (10:29 +0200)]
Added mypy testing

8 months agoAdd TODO for brighter days
Francesco Siddi [Thu, 16 Aug 2018 17:12:10 +0000 (19:12 +0200)]
Add TODO for brighter days

8 months agoAdd token revoke workflow
Francesco Siddi [Thu, 16 Aug 2018 17:09:35 +0000 (19:09 +0200)]
Add token revoke workflow

Introduce a new /token/revoke end point that accepts
POST requests with a token_id. If the request is
valid, the token is updated to is_valid=False.

Fixes T56380
Fixes T56399

8 months agoMove Blender ID OAuth client into named tuple
Francesco Siddi [Thu, 16 Aug 2018 13:47:51 +0000 (15:47 +0200)]
Move Blender ID OAuth client into named tuple

Does not crash the app when BLENDER_ID is
not configured!

8 months agoInclude the Django Admin docs
Sybren A. Stüvel [Thu, 16 Aug 2018 10:30:11 +0000 (12:30 +0200)]
Include the Django Admin docs

8 months agoDo version checking of the Benchmark Client
Sybren A. Stüvel [Thu, 16 Aug 2018 10:29:41 +0000 (12:29 +0200)]
Do version checking of the Benchmark Client

- The result JSON now includes the benchmark client version, and as a
  result the schema version has been bumped to 1. This version is not
  checked yet on the server, so it's at the moment purely cosmetic.
- Latest version of the Benchmark Client is configured in
  LATEST_CLIENT_VERSIONS['BENCHMARK']. This allows us to have different
  clients with different latest versions.
- An empty string as 'latest version' disables the version check altogether.
  This allows us to push to the production servers before we release the
  client version that actually includes the version information.

8 months agoShow timezone in 'Your Benchmarks'
Sybren A. Stüvel [Thu, 16 Aug 2018 08:48:10 +0000 (10:48 +0200)]
Show timezone in 'Your Benchmarks'

Showing time without timezone is meaningless.

8 months agoReformatted example JSON
Sybren A. Stüvel [Thu, 16 Aug 2018 08:47:45 +0000 (10:47 +0200)]
Reformatted example JSON

We use four spaces for JSON in all our projects.

8 months agoSplit views.py into separate files
Sybren A. Stüvel [Wed, 15 Aug 2018 14:21:03 +0000 (16:21 +0200)]
Split views.py into separate files

8 months agoBasic admin backend customization
Francesco Siddi [Wed, 15 Aug 2018 14:53:49 +0000 (16:53 +0200)]
Basic admin backend customization

Display site name in admin header templates.
Fixes T56400

8 months agoMatch Blender ID navigation
Francesco Siddi [Wed, 15 Aug 2018 13:30:33 +0000 (15:30 +0200)]
Match Blender ID navigation

8 months agoFix T56381: store hostname with client auth token
Sybren A. Stüvel [Wed, 15 Aug 2018 11:07:40 +0000 (13:07 +0200)]
Fix T56381: store hostname with client auth token

8 months agoChange default ordering of Benchmark and ClientToken to newest-first
Sybren A. Stüvel [Wed, 15 Aug 2018 11:07:14 +0000 (13:07 +0200)]
Change default ordering of Benchmark and ClientToken to newest-first

8 months agoUpdated webassets
Sybren A. Stüvel [Wed, 15 Aug 2018 10:53:18 +0000 (12:53 +0200)]
Updated webassets

8 months agoDouble quotes for consistency
Francesco Siddi [Wed, 15 Aug 2018 10:30:59 +0000 (12:30 +0200)]
Double quotes for consistency

8 months agoAdd favicon
Francesco Siddi [Wed, 15 Aug 2018 10:28:03 +0000 (12:28 +0200)]
Add favicon

8 months agoUpdate nginx configuration
Francesco Siddi [Wed, 15 Aug 2018 10:04:19 +0000 (12:04 +0200)]
Update nginx configuration

Now static files for opendata are located in /static/

8 months agoRemove unused config file
Francesco Siddi [Wed, 15 Aug 2018 09:13:30 +0000 (11:13 +0200)]
Remove unused config file

8 months agoConform directory layout
Francesco Siddi [Wed, 15 Aug 2018 09:13:17 +0000 (11:13 +0200)]
Conform directory layout

Move static to webstatic and src to websrc.
This way when we run ./manage.py collectstatic,
files will be copied to the standard static location.

Fixes T56402

8 months agoUpdate repo links to new official location
Francesco Siddi [Tue, 14 Aug 2018 14:27:16 +0000 (16:27 +0200)]
Update repo links to new official location

8 months agoAdd terms of service and privacy policy links
Pablo Vazquez [Tue, 14 Aug 2018 14:10:54 +0000 (16:10 +0200)]
Add terms of service and privacy policy links

8 months agoPadding mismatch with Blender ID
Pablo Vazquez [Tue, 14 Aug 2018 14:06:58 +0000 (16:06 +0200)]
Padding mismatch with Blender ID

8 months agoTweak prep_elastic.sh to run on macOS
Francesco Siddi [Fri, 10 Aug 2018 19:03:13 +0000 (21:03 +0200)]
Tweak prep_elastic.sh to run on macOS

8 months agoShow a nice message for a 502 Bad Gateway error
Sybren A. Stüvel [Fri, 10 Aug 2018 15:14:59 +0000 (17:14 +0200)]
Show a nice message for a 502 Bad Gateway error

8 months agoAuthentication -> Applications
Pablo Vazquez [Fri, 10 Aug 2018 15:03:56 +0000 (17:03 +0200)]
Authentication -> Applications

8 months agoAdded single do-everything deploy script
Sybren A. Stüvel [Fri, 10 Aug 2018 14:51:45 +0000 (16:51 +0200)]
Added single do-everything deploy script

8 months agoProper redirect for opendata http → https
Sybren A. Stüvel [Fri, 10 Aug 2018 14:38:58 +0000 (16:38 +0200)]
Proper redirect for opendata http → https

8 months agoMyData is in beta!
Pablo Vazquez [Fri, 10 Aug 2018 14:36:15 +0000 (16:36 +0200)]
MyData is in beta!

8 months agoTutti from BWA for navbar hiding
Pablo Vazquez [Fri, 10 Aug 2018 14:36:03 +0000 (16:36 +0200)]
Tutti from BWA for navbar hiding