blender-asset-tracer.git
7 days agoBumped version to 1.1.1 master v1.1.1
Sybren A. Stüvel [Thu, 18 Apr 2019 10:04:51 +0000 (12:04 +0200)]
Bumped version to 1.1.1

7 days agoImport urllib3 from requests.packages.urllib3
Sybren A. Stüvel [Thu, 18 Apr 2019 10:04:03 +0000 (12:04 +0200)]
Import urllib3 from requests.packages.urllib3

This is for compatibility with Blender 2.79.

7 days agoDeclare requests as dependency
Sybren A. Stüvel [Thu, 18 Apr 2019 09:47:28 +0000 (11:47 +0200)]
Declare requests as dependency

It's used by the Shaman interface only, so other BAT functions could be
used without it as well.

7 days agoAllow testing with Tox
Sybren A. Stüvel [Thu, 18 Apr 2019 09:43:18 +0000 (11:43 +0200)]
Allow testing with Tox

4 weeks agoBumped version to 1.1 v1.1
Sybren A. Stüvel [Mon, 25 Mar 2019 16:46:53 +0000 (17:46 +0100)]
Bumped version to 1.1

4 weeks agoMarked version 1.1 as released
Sybren A. Stüvel [Mon, 25 Mar 2019 16:46:38 +0000 (17:46 +0100)]
Marked version 1.1 as released

4 weeks agoUpdated URL to Shaman
Sybren A. Stüvel [Mon, 25 Mar 2019 16:46:28 +0000 (17:46 +0100)]
Updated URL to Shaman

The Shaman server as standalone component is no more, it's been integrated
into Flamenco Manager.

5 weeks agoBumped version to 1.1.dev2
Sybren A. Stüvel [Wed, 20 Mar 2019 12:58:23 +0000 (13:58 +0100)]
Bumped version to 1.1.dev2

5 weeks agoSupport Alembic files from linked library
Sybren A. Stüvel [Wed, 20 Mar 2019 12:56:03 +0000 (13:56 +0100)]
Support Alembic files from linked library

We already supported Alembic files in the top-level blend file, but now we
also support finding Alembic files in linked-in libraries.

5 weeks agoSupport the Mesh Sequence Cache modifier (a.k.a. Alembic files)
Sybren A. Stüvel [Wed, 20 Mar 2019 10:14:23 +0000 (11:14 +0100)]
Support the Mesh Sequence Cache modifier (a.k.a. Alembic files)

5 weeks agoFormatting
Sybren A. Stüvel [Wed, 20 Mar 2019 10:12:47 +0000 (11:12 +0100)]
Formatting

7 weeks agoUse exponential backoff when retrying HTTP connections
Sybren A. Stüvel [Wed, 6 Mar 2019 15:21:30 +0000 (16:21 +0100)]
Use exponential backoff when retrying HTTP connections

7 weeks agoShaman: added some timeouts to HTTP requests
Sybren A. Stüvel [Wed, 6 Mar 2019 15:00:46 +0000 (16:00 +0100)]
Shaman: added some timeouts to HTTP requests

7 weeks agoBumped version to 1.1-dev1
Sybren A. Stüvel [Wed, 6 Mar 2019 12:40:32 +0000 (13:40 +0100)]
Bumped version to 1.1-dev1

7 weeks agoFix ValueError when not requesting a checkout
Sybren A. Stüvel [Wed, 6 Mar 2019 11:10:55 +0000 (12:10 +0100)]
Fix ValueError when not requesting a checkout

7 weeks agoPython 3.5 fixes
Sybren A. Stüvel [Fri, 1 Mar 2019 14:12:24 +0000 (15:12 +0100)]
Python 3.5 fixes

`typing.Deque` was introduced in Python 3.6, so we can only use it when
we no longer support Blender 2.79.

7 weeks agoUpdated changelog
Sybren A. Stüvel [Fri, 1 Mar 2019 13:32:50 +0000 (14:32 +0100)]
Updated changelog

7 weeks agoShaman: Include original filename when uploading file
Sybren A. Stüvel [Fri, 1 Mar 2019 12:54:16 +0000 (13:54 +0100)]
Shaman: Include original filename when uploading file

7 weeks agoShaman changed response `does-not-exist` → `file-unknown`
Sybren A. Stüvel [Thu, 28 Feb 2019 11:34:23 +0000 (12:34 +0100)]
Shaman changed response `does-not-exist` → `file-unknown`

Instead of 'does-not-exist' we'll now receive 'file-unknown' (see Shaman
commit 44efa6b39bddc703a14aa68316b092866813c38a)

7 weeks agoShaman: Moved URL parsing into a separate function
Sybren A. Stüvel [Thu, 28 Feb 2019 11:32:07 +0000 (12:32 +0100)]
Shaman: Moved URL parsing into a separate function

This allows other code to just call that function, rather than copying
the behaviour.

7 weeks agoWhen aborting, make sure the reason is available
Sybren A. Stüvel [Thu, 28 Feb 2019 11:28:57 +0000 (12:28 +0100)]
When aborting, make sure the reason is available

This allows a GUI to display the reason of abortion.

7 weeks agoShaman: Make checkout location available to the Packer
Sybren A. Stüvel [Wed, 27 Feb 2019 17:26:20 +0000 (18:26 +0100)]
Shaman: Make checkout location available to the Packer

7 weeks agoRecord bytes & files uploaded on Transferer
Sybren A. Stüvel [Wed, 27 Feb 2019 15:22:28 +0000 (16:22 +0100)]
Record bytes & files uploaded on Transferer

7 weeks agoShaman server support
Sybren A. Stüvel [Tue, 26 Feb 2019 14:26:40 +0000 (15:26 +0100)]
Shaman server support

The Shaman server is a file storage system that identifies files by
SHA256sum and file length. BAT can send packs there by only uploading
changed/new files. The BAT pack is reproduced at the Shaman server's
checkout directory by creating symlinks to the files in its file
storage.

Retrying sending files:

When we can defer uploading a file (that is, when we have other files to
upload as well, and we could send the current file at a later moment) we
send an `X-Shaman-Can-Defer-Upload: true` header in the file upload
request. In that case, when someone else is already uploading that file,
a `208 Already Reported` response is sent and the connection is closed.
Python's Requests library unfortunately won't give us that response if
we're still streaming the request, and raise a ConnectionError exception
instead. This exception can mean two things:

- If the `X-Shaman-Can-Defer-Upload: true` header was sent: someone else
  is currently uploading that file, so defer it.
- If that header was not sent: that file is already completely uploaded
  and does not need to be uploaded again.

Instead of retrying each failed file, after a few failures we now just
resend the definition file to get a new list of files to upload, then
send those. This should considerably reduce the number of HTTP calls
when multiple clients are uploading the same set of files.

7 weeks agoBumped version to 1.1-dev0
Sybren A. Stüvel [Fri, 1 Mar 2019 13:07:12 +0000 (14:07 +0100)]
Bumped version to 1.1-dev0

7 weeks agoBumped version to 1.0 v1.0
Sybren A. Stüvel [Fri, 1 Mar 2019 13:03:26 +0000 (14:03 +0100)]
Bumped version to 1.0

This is the same version as 0.99, just released as 1.0 to note the
maturity of the code (it's been used successfully for months at the Blender
Animation Studio).

8 weeks agoBumped version to 0.99 v0.99
Sybren A. Stüvel [Tue, 26 Feb 2019 15:45:43 +0000 (16:45 +0100)]
Bumped version to 0.99

8 weeks agoInclude error message with error status
Sybren A. Stüvel [Tue, 26 Feb 2019 15:20:05 +0000 (16:20 +0100)]
Include error message with error status

8 weeks agoConvert target path from Path to str & PurePath
Sybren A. Stüvel [Tue, 26 Feb 2019 14:22:14 +0000 (15:22 +0100)]
Convert target path from Path to str & PurePath

The target path is just read as string from the CLI now, to allow more
complex targets (such as URLs) that don't directly map to a path.

The Packer subclass now handles the conversion from that string to a
`pathlib.PurePath`, and specific subclasses & transfer classes can convert
those to a `pathlib.Path` to perform actual filesystem operations when
necessary.

8 weeks agoUpdate dependencies
Sybren A. Stüvel [Tue, 26 Feb 2019 14:17:59 +0000 (15:17 +0100)]
Update dependencies

8 weeks agoUse `'bat-'` prefix when creating temporary directories
Sybren A. Stüvel [Tue, 26 Feb 2019 14:33:42 +0000 (15:33 +0100)]
Use `'bat-'` prefix when creating temporary directories

8 weeks agoRemoved unused imports
Sybren A. Stüvel [Tue, 26 Feb 2019 13:49:54 +0000 (14:49 +0100)]
Removed unused imports

2 months agoClarified comment
Sybren A. Stüvel [Tue, 12 Feb 2019 11:48:02 +0000 (12:48 +0100)]
Clarified comment

2 months agoBumped version to 0.9
Sybren A. Stüvel [Tue, 12 Feb 2019 11:31:51 +0000 (12:31 +0100)]
Bumped version to 0.9

2 months agoFixed crash where collection children are ID blocks instead of GR blocks
Sybren A. Stüvel [Tue, 12 Feb 2019 11:31:05 +0000 (12:31 +0100)]
Fixed crash where collection children are ID blocks instead of GR blocks

Not sure when/how that happens, but it happened on a lighting file of the
Spring project, when linking in a nested collection of which parts where
also linked in from other blend files.

2 months agoAdded some more unit tests for BlendPath
Sybren A. Stüvel [Mon, 4 Feb 2019 14:42:40 +0000 (15:42 +0100)]
Added some more unit tests for BlendPath

2 months agoRemoved unused FileCopierPool class
Sybren A. Stüvel [Mon, 4 Feb 2019 14:34:29 +0000 (15:34 +0100)]
Removed unused FileCopierPool class

2 months agoAdded --sha256 and --timing options to `bat list`
Sybren A. Stüvel [Mon, 4 Feb 2019 14:16:35 +0000 (15:16 +0100)]
Added --sha256 and --timing options to `bat list`

These are only temporary features for aiding in development.

2 months agoRemoved some extraneous commas
Sybren A. Stüvel [Mon, 4 Feb 2019 14:15:43 +0000 (15:15 +0100)]
Removed some extraneous commas

2 months agoUse a priority queue instead of repeatedly sorting a deque
Sybren A. Stüvel [Fri, 1 Feb 2019 17:03:25 +0000 (18:03 +0100)]
Use a priority queue instead of repeatedly sorting a deque

3 months agoBumped version to 0.8 v0.8
Sybren A. Stüvel [Wed, 2 Jan 2019 15:06:38 +0000 (16:06 +0100)]
Bumped version to 0.8

3 months agoTest packing a blend file with Windows relative paths
Sybren A. Stüvel [Wed, 2 Jan 2019 14:41:56 +0000 (15:41 +0100)]
Test packing a blend file with Windows relative paths

3 months agoAvoid repeating/constructing the same path
Sybren A. Stüvel [Wed, 2 Jan 2019 14:39:09 +0000 (15:39 +0100)]
Avoid repeating/constructing the same path

3 months agoConfigure test logging at import time (rather than at setup time)
Sybren A. Stüvel [Wed, 2 Jan 2019 14:38:34 +0000 (15:38 +0100)]
Configure test logging at import time (rather than at setup time)

3 months agoWorkaround for Windows failing on glob patterns in path.resolve()
Sybren A. Stüvel [Wed, 2 Jan 2019 14:04:54 +0000 (15:04 +0100)]
Workaround for Windows failing on glob patterns in path.resolve()

Windows fails with an OSError when `somepath.resolve()` is called and
`somepath` contains a glob pattern. As a workaround, we now `resolve()`
the parent directory, and put the filename at its end. This only works
when the glob pattern is in the filename, which is the case for BAT-
generated globs.

3 months agoExplicitly write pack-info.txt as UTF-8
Sybren A. Stüvel [Wed, 2 Jan 2019 14:03:26 +0000 (15:03 +0100)]
Explicitly write pack-info.txt as UTF-8

3 months agoReplace → with ->
Sybren A. Stüvel [Wed, 2 Jan 2019 13:32:10 +0000 (14:32 +0100)]
Replace → with ->

Apparently Windows still isn't using UTF-8 by default, and fails when
handling these log entries.

3 months agoMissing test teardowns
Sybren A. Stüvel [Wed, 2 Jan 2019 11:47:43 +0000 (12:47 +0100)]
Missing test teardowns

This kept a file pointer open while trying to delete the file.

3 months agoBlendPath: don't use os.path.join() for slash notations
Sybren A. Stüvel [Wed, 2 Jan 2019 11:47:16 +0000 (12:47 +0100)]
BlendPath: don't use os.path.join() for slash notations

The BlendPath should just use forward slashes, and not be depending on the
current platform.

3 months agoBlendPath.__repr__ implemented
Sybren A. Stüvel [Wed, 2 Jan 2019 11:46:33 +0000 (12:46 +0100)]
BlendPath.__repr__ implemented

3 months agoBlendPath.as_path() now refuses to convert blendfile-relative paths
Sybren A. Stüvel [Thu, 20 Dec 2018 14:35:06 +0000 (15:35 +0100)]
BlendPath.as_path() now refuses to convert blendfile-relative paths

When a path starts with b'//' it will not be converted to a PurePath, as
the handling of such filenames is platform dependent (Windows handles those
weirdly, like appending a slash to any path).

3 months agoAttempt at making BAT work on Windows
Sybren A. Stüvel [Thu, 20 Dec 2018 14:15:49 +0000 (15:15 +0100)]
Attempt at making BAT work on Windows

All BlendPath instances will use forward slashes, and there should be
more use of PurePosixPath instead of Path.

4 months agoUpgraded dependencies
Sybren A. Stüvel [Thu, 20 Dec 2018 13:23:12 +0000 (14:23 +0100)]
Upgraded dependencies

4 months agoFixed MyPy issue
Sybren A. Stüvel [Thu, 20 Dec 2018 13:23:06 +0000 (14:23 +0100)]
Fixed MyPy issue

4 months agoFixed leaking file descriptors in unit tests
Sybren A. Stüvel [Thu, 20 Dec 2018 13:04:23 +0000 (14:04 +0100)]
Fixed leaking file descriptors in unit tests

These weren't any issue in Linux, but Windows doesn't like it when you
try to delete a still-opened file.

4 months agoBumped version to 0.7 v0.7
Sybren A. Stüvel [Thu, 6 Dec 2018 09:02:40 +0000 (10:02 +0100)]
Bumped version to 0.7

4 months agoAdded `--relative-only` option to `bat pack` command
Sybren A. Stüvel [Thu, 6 Dec 2018 09:02:14 +0000 (10:02 +0100)]
Added `--relative-only` option to `bat pack` command

This makes BAT skip assets that are referred to with an absolute path.
It is assumed that the receiver of the BAT pack can access those assets
at the same path.

4 months agoBumped version to 0.6.1 v0.6.1
Sybren A. Stüvel [Fri, 30 Nov 2018 13:03:31 +0000 (14:03 +0100)]
Bumped version to 0.6.1

4 months agoblocks: Allow choosing number of DNA types shown via --limit CLI arg
Sybren A. Stüvel [Fri, 30 Nov 2018 13:03:20 +0000 (14:03 +0100)]
blocks: Allow choosing number of DNA types shown via --limit CLI arg

4 months agoBumped version to 0.6 v0.6
Sybren A. Stüvel [Fri, 30 Nov 2018 12:59:33 +0000 (13:59 +0100)]
Bumped version to 0.6

4 months agoAdded 'bat blocks' command that shows the DNA types that use the most data
Sybren A. Stüvel [Fri, 30 Nov 2018 12:58:19 +0000 (13:58 +0100)]
Added 'bat blocks' command that shows the DNA types that use the most data

It's work in progress and maybe not useful for anyone but us.

4 months agoAdded Twine to dev requirements v0.5
Sybren A. Stüvel [Tue, 27 Nov 2018 15:06:51 +0000 (16:06 +0100)]
Added Twine to dev requirements

Twine is the recommended way to upload files to PyPi.

4 months agoTypo
Sybren A. Stüvel [Tue, 27 Nov 2018 15:00:50 +0000 (16:00 +0100)]
Typo

4 months agoAdded instructions on how to upload packages to pypi
Sybren A. Stüvel [Tue, 27 Nov 2018 14:59:58 +0000 (15:59 +0100)]
Added instructions on how to upload packages to pypi

4 months agoRemoved logging options from setup.cfg again
Sybren A. Stüvel [Tue, 27 Nov 2018 14:59:43 +0000 (15:59 +0100)]
Removed logging options from setup.cfg again

The % symbols confuse setup.py when running 'setup.py sdist bdist_wheel',
and I couldn't quickly figure out how to escape those properly.

4 months agoBumped version to 0.5
Sybren A. Stüvel [Tue, 27 Nov 2018 14:51:19 +0000 (15:51 +0100)]
Bumped version to 0.5

4 months agoMulti-threaded compressed file transfers
Sybren A. Stüvel [Tue, 27 Nov 2018 14:50:20 +0000 (15:50 +0100)]
Multi-threaded compressed file transfers

For regular file transfers (so to a directory, not to a ZIP file or S3
storage), use multi-threaded transfer when compressing. Compressing is
CPU-bound, so using multiple threads speeds things up considerably
(packing a Spring lighting file went from 6min30 single-threaded to
2min13 multi-threaded on my machine).

4 months ago`bat pack --help` improvements
Sybren A. Stüvel [Tue, 27 Nov 2018 13:37:15 +0000 (14:37 +0100)]
`bat pack --help` improvements

4 months agoAdded --compress option for 'bat pack' command
Sybren A. Stüvel [Tue, 27 Nov 2018 13:20:08 +0000 (14:20 +0100)]
Added --compress option for 'bat pack' command

This compresses all packed Blend files. Other files, as well as already-
compressed Blend files, are left as-is.

4 months agoAdd some log level configuration to py.test config
Sybren A. Stüvel [Tue, 27 Nov 2018 13:21:13 +0000 (14:21 +0100)]
Add some log level configuration to py.test config

Not sure if we want a global DEBUG setting; if it becomes annoying, change it.

5 months agoSupport for cloth ptcaches stored on disk
Jeroen Bakker [Thu, 8 Nov 2018 09:25:02 +0000 (10:25 +0100)]
Support for cloth ptcaches stored on disk

Currently pointcaches on disk for cloth sims were not traced. This patch allows the tracing of pointcaches for cloth sims.

Reviewers: sybren, fsiddi

Reviewed By: sybren

Differential Revision: https://developer.blender.org/D3910

5 months agoFix MyPy errors when running without boto3 installed
Sybren A. Stüvel [Wed, 7 Nov 2018 12:03:07 +0000 (13:03 +0100)]
Fix MyPy errors when running without boto3 installed

9 months agoInclude version in --help output
Sybren A. Stüvel [Wed, 11 Jul 2018 10:44:29 +0000 (12:44 +0200)]
Include version in --help output

9 months agoBumped version to 0.5-dev
Sybren A. Stüvel [Tue, 10 Jul 2018 14:03:30 +0000 (16:03 +0200)]
Bumped version to 0.5-dev

9 months agoBumped version to 0.4 v0.4
Sybren A. Stüvel [Tue, 10 Jul 2018 14:03:00 +0000 (16:03 +0200)]
Bumped version to 0.4

9 months agoBlender 2.8 compatibility of image datablock
Sybren A. Stüvel [Tue, 10 Jul 2018 13:37:46 +0000 (15:37 +0200)]
Blender 2.8 compatibility of image datablock

9 months agoMoved from requirements.txt to pipenv for package management
Sybren A. Stüvel [Tue, 10 Jul 2018 13:48:19 +0000 (15:48 +0200)]
Moved from requirements.txt to pipenv for package management

9 months agoFixed a whole bunch of MyPy errors
Sybren A. Stüvel [Tue, 10 Jul 2018 13:44:52 +0000 (15:44 +0200)]
Fixed a whole bunch of MyPy errors

New release of mypy picks up more.

9 months agoFix T55542: BAT doesn't pack smoke cache
Philipp Oeser [Thu, 5 Jul 2018 07:33:37 +0000 (09:33 +0200)]
Fix T55542: BAT doesn't pack smoke cache

9 months agoConvert hexified ID name to upper case
Sybren A. Stüvel [Thu, 5 Jul 2018 09:53:38 +0000 (11:53 +0200)]
Convert hexified ID name to upper case

Our test case accidentally has a numerical-only hex representation, which
doesn't show this issue.

9 months agoBumped version to 0.4-dev
Sybren A. Stüvel [Tue, 3 Jul 2018 13:07:34 +0000 (15:07 +0200)]
Bumped version to 0.4-dev

9 months agoBumped version to 0.3 v0.3
Sybren A. Stüvel [Tue, 3 Jul 2018 13:06:27 +0000 (15:06 +0200)]
Bumped version to 0.3

9 months agoFix T55539: BAT doesn't pack particle caches
Sybren A. Stüvel [Tue, 3 Jul 2018 12:41:12 +0000 (14:41 +0200)]
Fix T55539: BAT doesn't pack particle caches

9 months agoTracer fix for T55539
Sybren A. Stüvel [Tue, 3 Jul 2018 12:33:28 +0000 (14:33 +0200)]
Tracer fix for T55539

This only fixes the tracer; the problem with the packer still exists.

9 months agoSwap 'expected' and 'actual' arguments
Sybren A. Stüvel [Tue, 3 Jul 2018 12:30:59 +0000 (14:30 +0200)]
Swap 'expected' and 'actual' arguments

Python itself doesn't make any distinction (it just logs 'a != b') but
PyCharm does.

9 months agoUnignore some paths
Sybren A. Stüvel [Tue, 3 Jul 2018 12:00:53 +0000 (14:00 +0200)]
Unignore some paths

9 months agoFlush the Packer callback queue when closing
Sybren A. Stüvel [Tue, 3 Jul 2018 11:01:36 +0000 (13:01 +0200)]
Flush the Packer callback queue when closing

9 months agoReport skipped files
Sybren A. Stüvel [Tue, 3 Jul 2018 11:00:10 +0000 (13:00 +0200)]
Report skipped files

9 months agoAlso report transfer progress on MOVE
Sybren A. Stüvel [Tue, 3 Jul 2018 10:56:08 +0000 (12:56 +0200)]
Also report transfer progress on MOVE

Previously transfers were only reported when copying files.

9 months agoAdded utility function to fetching structs by name
Sybren A. Stüvel [Tue, 3 Jul 2018 10:34:00 +0000 (12:34 +0200)]
Added utility function to fetching structs by name

10 months agoSupport nested collections in Blender 2.8
Sybren A. Stüvel [Wed, 13 Jun 2018 15:40:05 +0000 (17:40 +0200)]
Support nested collections in Blender 2.8

10 months agoBlender 2.8 fixes because of removed Material.mtex and Material.group
Sybren A. Stüvel [Thu, 7 Jun 2018 13:18:18 +0000 (15:18 +0200)]
Blender 2.8 fixes because of removed Material.mtex and Material.group

10 months agoFix T55007: Support fluid simulation cache files
Sybren A. Stüvel [Tue, 5 Jun 2018 08:33:00 +0000 (10:33 +0200)]
Fix T55007: Support fluid simulation cache files

This also adds support for entire directory paths to be assets, in
addition to globs and numbered file sequences.

10 months agoBetter transfer error handling
Sybren A. Stüvel [Tue, 5 Jun 2018 09:49:55 +0000 (11:49 +0200)]
Better transfer error handling

10 months agoSpeed up packing
Sybren A. Stüvel [Tue, 5 Jun 2018 12:47:24 +0000 (14:47 +0200)]
Speed up packing

The .get() function raises a queue.Empty exception anyway, so there was
no real need for the .empty() call. Furthermore, it turned out to really
slow things down.

10 months agoAdded missing image in documentation
Sybren A. Stüvel [Tue, 5 Jun 2018 13:11:19 +0000 (15:11 +0200)]
Added missing image in documentation

11 months agoBumped version to 0.3-dev
Sybren A. Stüvel [Tue, 8 May 2018 10:51:00 +0000 (12:51 +0200)]
Bumped version to 0.3-dev

11 months agoBumped version to 0.2 v0.2
Sybren A. Stüvel [Tue, 8 May 2018 10:50:37 +0000 (12:50 +0200)]
Bumped version to 0.2

11 months agoDebug-log skipping of packed datablocks.
Sybren A. Stüvel [Tue, 8 May 2018 10:15:56 +0000 (12:15 +0200)]
Debug-log skipping of packed datablocks.

12 months agoAdded BAT logo to documentation
Sybren A. Stüvel [Tue, 27 Mar 2018 10:28:34 +0000 (12:28 +0200)]
Added BAT logo to documentation