6c77ddd1c3
searchhelpers.normalize_author use un only, not IDs.
...
I don't like the idea of a user setting their username to be
someone else's ID.
2018-03-21 13:21:25 -07:00
afc5f38a8d
Combine normalize_limit, _offset to normalize_positive_integer.
2018-03-21 11:25:21 -07:00
1d76b4511b
Let PDB.get_tag convert normalization errors to NoSuch.
...
Instead of having to account for them on the other end,
since they don't care about that stuff anyway.
2018-03-21 11:03:58 -07:00
9a1caee412
Add helper function hash_file.
2018-03-18 20:56:29 -07:00
2d73e59abf
Let helpers.read_filebytes operate on pathclass Paths.
2018-03-18 20:56:08 -07:00
e186653e36
Use shutil.which to discover ffmpeg.
...
How did I miss such an easy solution for so long??
So glad I did not go the env var route.
2018-03-18 17:03:11 -07:00
84e0266f30
Add author to object jsonifiers.
2018-03-18 15:59:36 -07:00
60049c777f
Add method get_author to all objects.
2018-03-18 15:58:51 -07:00
dffde094e8
Give Albums and Tags an author_id column.
...
Keeping V11 because I hadn't published previous commits yet.
2018-03-18 15:28:26 -07:00
d6d28764c4
Oops, assign self.config before dumping it.
2018-03-18 00:23:05 -07:00
6add02c8a2
Move column extractor to own function.
2018-03-18 00:21:03 -07:00
d383000441
Add Foreign Keys, Primary Keys, and NOT NULL constraints.
2018-03-18 00:09:08 -07:00
33d506ddaf
Rearrange tables in preparation for foreign keys.
...
In order to create a foreign key the table has to already exist.
So move the object types to the top, with Users first since other
things have authors.
2018-03-17 17:54:15 -07:00
c4d0de7f97
Move digest_directory and easybake to Util mixin.
2018-03-17 17:19:07 -07:00
87886a0a13
Unprivatize PDB._load_config and _save_config.
2018-03-17 17:07:30 -07:00
941d50eae0
Move config saving to own method.
2018-03-17 17:05:34 -07:00
269cfb6762
Improve some variable names for clarity.
2018-03-17 16:48:09 -07:00
ffe0be21f5
Fix Photo thumbnail paths to be relative to the PDB's thumb dir.
...
Previously they were relative to the directory from which the client
was launched, meaning it usually started with ./_etiquette.
Now it should be more portable and less tight.
2018-03-13 02:50:54 -07:00
44d1fb2fff
Add PhotoDB init option skip_version_check.
2018-03-13 02:25:04 -07:00
d85d911b22
Add PhotoDB init option create. Disable to require existing.
2018-03-13 02:24:11 -07:00
101f6d9c47
Add exception BadDataDirectory.
2018-03-13 02:21:35 -07:00
b89da5dec2
Add hasattr checks to prevent faulty __del__.
2018-03-13 02:20:17 -07:00
1cd78a678b
Minor linter appeasements.
2018-03-11 01:54:59 -08:00
c99e783d04
Rename comprehension variable to not look like a conflict.
...
The existing code worked fine, but it was odd to see that `tag`
was both the loop variable as well as the key into the children.
So let's change it.
2018-03-11 01:35:35 -08:00
a86c27ceb4
Simplify some unnecessarily detailed log formats.
2018-03-11 01:34:32 -08:00
94e518068f
Add Album.add_photos for batch adding with less savepoint waste.
...
Because each call to add_photo opens a savepoint, it is wasteful
to use it for a large number of photos that belong together.
2018-03-10 22:50:41 -08:00
f5a126d3bc
Oops, fix missing return statement.
2018-03-10 15:42:23 -08:00
42cef093a5
digest_directory: Use _normalize functions for argument prep.
...
This is a pattern I want to try out.
Not sure if it's ugly yet.
2018-03-10 15:32:31 -08:00
633dbfa3cf
Fix typo, searchhelpers.py is a file.
2018-03-10 14:42:33 -08:00
9355cd09ef
Add digest_directory parameter new_photo_ratelimit.
...
Initially inspired by the nuisance of having multiple photos
with the same timestamp because the system is too fast, this
feature could also be used to lessen disk load for running in
the background.
2018-03-10 14:18:28 -08:00
c90585665a
Add md and rst as text/plain mimetypes.
2018-03-10 13:27:21 -08:00
5f6d21fdee
Give Photos a searchhidden property.
...
By default, photos with searchhidden do not appear in the search
results. This allows a small number of representative images from
a large album to appear in the results, while the rest can be
found on the album's page.
The same effect could be achieved with a tag and forbid search,
but tag searching has much higher cost and it would be more difficult
to implement as a default behavior without requiring lots of special
checks whenever listing tags etc.
2018-03-09 17:10:27 -08:00
d88db08693
Allow PDB.rollback to take a savepoint; Warn early commits.
...
If a @transaction method makes calls to other @transaction
methods, and one of those nested calls makes a commit, and then
the outer call raises an exception, then the outer call will
not rollback properly because its savepoint is no longer in
the savepoint stack. So let's warn the user if that happens.
Should this raise an exception instead of just warn? Not sure,
I mean the data is already committed.
2018-03-05 21:35:36 -08:00
26141f8198
Put savepoints in the on_commit_queue so we can cancel on rb.
2018-03-02 20:30:55 -08:00
8b9ef2a492
Oops, add commit parameter to Photo.copy_tags.
2018-02-25 18:56:11 -08:00
4ae5f52db3
Add some missing decorators and explain others.
...
Leave comments in places where @transaction or @required_feature
might be expected, but are taken care of by something else.
2018-02-25 18:55:46 -08:00
3d925c1426
Rename inner wrapped to improve traceback clarity.
...
Because "line 77, in wrapped_transaction" is at least slightly
better than "in wrapped".
2018-02-25 17:15:35 -08:00
c001b0fc87
Use SQL savepoints to improve @transaction.
...
I didn't know about these! Now we don't have to roll all the way
back in case of problems. This allows for raising exceptions as
control flow since they won't undo unrelated stuff.
2018-02-25 17:10:04 -08:00
576fac416a
Move SQL heavy lifting to its own Mixin.
2018-02-25 16:30:05 -08:00
e4e542a8a6
Replace manual DELETEs with calls to sql_delete.
2018-02-25 16:17:19 -08:00
a2e13003e8
Add PhotoDB.sql_delete.
2018-02-25 16:09:36 -08:00
f34164bf85
Add search feature has_thumbnails.
...
Search needs a complete refactor.
But until then, let's keep adding to it!
2018-02-24 23:06:25 -08:00
d653673277
Update truthystring to return bool, not given int.
2018-02-24 23:01:44 -08:00
0228fbebfd
Move the random hex generator to helpers.
2018-02-24 18:54:59 -08:00
13e8bc4a6f
Add more items to to-do list; minor fixes.
2018-02-20 16:50:39 -08:00
5d6c5ef2b4
Raise exception if attempted override name is invalid.
...
Technically it doesn't really matter because the override is not
used for any file ops, but I don't like the idea of busted
path-looking strings being displayed anywhere on the interface.
2018-02-17 15:47:26 -08:00
606a66cbf9
Remove Photo.filepath and real_filepath in favor of real_path.
2018-02-17 15:19:36 -08:00
42728d5d67
Add method Photo.set_override_filename.
2018-02-17 15:07:26 -08:00
f7bd49b2dd
Add call to remove_control_chars in remove_bad_pathchars.
2018-02-17 15:07:12 -08:00
0d6faa8f6e
Take advantage of new sql_update to update where_keys.
2018-02-17 14:02:11 -08:00
9939f5da1d
For sql_insert / _update, let default commit=False.
...
Because all the calls are using it that way anyway.
2018-02-17 14:01:12 -08:00
6363cb5fc0
Rename Album.associated_directories -> get_associated_directories.
2018-02-17 13:33:27 -08:00
b4a28f5034
Make Photo.bytestring a @property.
...
This is the kind of thing that should be a @property, because
it's very low processing power and doesn't hit the database.
2018-02-16 23:16:02 -08:00
bbe392ac09
Rename Photo.author -> get_author; no longer a @property.
2018-02-16 23:11:25 -08:00
205c49c21f
Rename Photo.albums -> get_containing_albums.
2018-02-16 23:08:44 -08:00
2f9b51b636
Rename Photo.tags -> get_tags.
2018-02-16 23:07:21 -08:00
ac9d7ede22
Rename Album.photos -> get_photos.
2018-02-16 23:03:54 -08:00
6574450ad1
Rename Tag.synonyms -> get_synonyms.
2018-02-16 23:01:07 -08:00
fdcf07940a
Rename Groupable.children -> get_children.
2018-02-16 23:00:43 -08:00
75effafaf5
Rename Groupable.parent -> get_parent.
...
After making the "if x.parent is None" mistake too many times,
forgetting that it was not a @property, I've decided that the
verby name is better.
2018-02-16 22:59:34 -08:00
db92951ad6
Oops, remove incorrect name= from new_tag.
2018-02-16 22:57:35 -08:00
3c4f69f647
Specify name= on calls to PDB.get_tag.
...
Sometimes it's easy to forget whether its the name or ID,
and I like the consistency of specifying.
2018-02-16 22:25:56 -08:00
16ac6f8b85
Move UN / PW validation to helper _assert methods.
2018-02-16 22:18:19 -08:00
173c07a0bf
Add some more exception hierarchy. Exists / InvalidUN/PW.
2018-02-16 22:13:54 -08:00
1c1e7f158f
Minor fixes & pylint appeasements.
2018-02-16 21:28:36 -08:00
e5a316ff9c
Remove old SQL_ constants. Use the dicts.
2018-02-16 20:40:58 -08:00
a8ead9c0c9
Replace a lot of manual insert/update with helper.
2018-02-16 20:19:18 -08:00
3e5031a1c5
Add methods sql_insert, sql_update to centralize sqlhelper work.
2018-02-16 18:49:02 -08:00
655d1fa235
Prepare to use dicts SQL_COLUMNS, _INDEX instead of separate.
2018-02-16 18:48:03 -08:00
4e8a67aa54
Avoid sql indexing by instantiating User earlier.
2018-02-16 18:40:57 -08:00
bc4e228a25
Remove a lot of the commented code from earlier search.
2018-02-04 15:11:46 -08:00
f7dcb489c5
Remove unnecessary intification of timestamps.
...
I decided I want to have higher precision especially because
batch inserts were having several photos with the same
created timestamp making orderby >= dumb.
2018-02-03 01:35:32 -08:00
dec28b321a
Rename recursive_dict_update's parameters for clarity.
2018-02-03 01:10:54 -08:00
f266e1c79c
Split PhotoDB's large init into some smaller portions.
2018-02-03 01:10:04 -08:00
f9ee56187c
Rename attr database_file to database_filepath.
...
To make it more consistent with config_filepath and show
that it's a path object.
2018-01-30 19:46:43 -08:00
c62e6cd80b
Fix synonym fetcher using outdated column name.
2018-01-13 20:02:49 -08:00
14750eafc5
Move DB_INIT to constants; auto parse column names.
2018-01-13 16:48:48 -08:00
178a7df0b3
Add some more log debugs.
2018-01-11 19:32:15 -08:00
b5902ba4f1
Integrate voussoirkit sqlhelpers.
...
https://github.com/voussoir/else/tree/master/SQLHelpers
2018-01-09 21:15:50 -08:00
6f371701e4
Fix digest second-scan bug; Break down into smaller functions.
...
There was a bug where moving an album out of its determined parent
caused future digests to fail because I wanted to use the GroupExists
for control flow, but due to the @transaction decorator it was rolling
back the changes.
Moved some of the logical pieces of the function into subfunctions
as a visual and readability aid.
2018-01-09 18:59:15 -08:00
d548e09a96
Add .nfo to the custom mimetypes as text/plain.
2018-01-09 18:46:32 -08:00
b859dbd22a
Add bookmark cache, replace get_bookmark code with call to get_thing.
2017-12-20 20:10:58 -08:00
0b5f736574
Few minor text fixes.
2017-12-16 03:45:07 -08:00
13803dc7f7
Oops, put remove_path_badchars where it belongs alphabetically.
2017-12-07 21:37:55 -08:00
f39717c216
Minor improvements to other helpers and their docstrings.
2017-12-07 21:32:37 -08:00
0d0354f4da
Rename normalize_filepath to remove_path_badchars.
...
Because the function does not do any resolving or converting
to an absolute path, I feel the name 'normalize' is inaccurate
and could be misleading when read. This is more clear.
2017-12-07 21:15:10 -08:00
6cf355d7ce
Rename comma_split to comma_space_split to better describe behavior.
2017-12-07 20:58:06 -08:00
a7b5a7d42a
Multiple small cleanups, docstrings, messages, lint improvements.
2017-11-27 15:56:16 -08:00
e6a50500b9
Fix missing file_read_chunk from default config
2017-11-26 02:37:33 -08:00
f052858d5f
Improve log creation and use custom formatter.
2017-11-26 02:37:11 -08:00
be35f3bb1f
Fix Tag.rename from raising TagExists for old instead of desired new.
2017-11-23 21:58:38 -08:00
64d8410977
No need for self.cur, just cur.
2017-11-16 18:03:51 -08:00
259c9ee1ab
Add PhotoDB init arg ephemeral. Uses :memory: sql and tempdir.
...
Primarily necessitated by unit testing. Running through the DB_INIT
is quite slow on disk, so this argument causes the sql to be done
on an in-memory database and all the other files are put into a
TemporaryDirectory.
Eventually I would like to have the other files be in-memory too
but that may be overcomplicated and underuseful.
2017-11-16 18:02:05 -08:00
74f4e74bdf
Add a few docstrings and update readme to-do list.
2017-11-16 16:46:39 -08:00
8f000543ea
Add exceptions.DatabaseOutOfDate instead of just using an error string.
...
And don't raise SystemExit. Not sure what I was thinking.
2017-11-16 16:33:40 -08:00
009c0dc678
Allow the data_directory to contain colons (fully qualified paths)
2017-11-16 16:32:05 -08:00
73d2ef5bbf
Change exception __str__ to be the type + message
...
Because the repr is ugly
2017-11-16 16:31:10 -08:00
a510c7b55c
Improve Album bytes caching, start caching photo count.
...
More careful uncaching of the summed bytes, to minimize
recalculation. Fewer cases where the album itself
is removed from the photodb's getter cache.
This also helps the download link on album pages disappear if
the child albums don't actually have any photos.
2017-11-12 15:21:53 -08:00
96856e9361
Cleanup: More minor dusting, commenting, clarity renaming.
2017-11-11 22:49:03 -08:00
bb5fa816d8
Cleanup: Rename GroupableMixin.add to add_child for clarity.
...
Especially since for albums it might sound like you're adding a photo.
2017-11-11 22:41:26 -08:00
3f69a2c240
Cleanup: Replace "objectid" with "object_id" in the Python portions.
2017-11-11 22:38:51 -08:00
a4b875b9ff
Fix config file not being updated when the default updates.
...
The len check was incorrect ever since I started nesting keys.
Now we measure all of the keys recursively.
2017-10-05 18:39:34 -07:00
a0f68fa858
Add .opus to the extended mimetypes.
2017-10-04 16:54:28 -07:00
121f8329fb
improve normalize_authors text
2017-09-29 14:32:35 -07:00
0849796986
Make build_query params all optional, more kwargy.
2017-09-23 11:22:05 -07:00
855e94ed87
Make wheres a set instead of list.
2017-09-23 11:20:38 -07:00
163e960bfa
Fix unrecognized orderby not using the warning bag.
2017-09-23 11:08:19 -07:00
557f78eb02
Integrate author search into the query instead of postprocessing.
...
Much faster this way.
2017-09-23 11:04:26 -07:00
38e72c0c3a
Make notnulls set instead of list.
2017-09-23 11:00:37 -07:00
26a034c9a0
Make Photo.author a @property; None if no author.
2017-09-23 10:58:04 -07:00
89d9c1d893
Move the new photo creation out of the exception handler.
...
I don't like the "during the handling of the above exception" because
it's misleading as to the source of the problem.
2017-08-01 18:13:54 -07:00
bd847159fa
Organize config by nesting related stuff.
...
I thought about switching to .ini files but that's even less desirable.
2017-07-29 16:23:15 -07:00
4fc203634c
Add new config "enable_login".
2017-07-20 23:25:23 -07:00
1784e584e2
Don't let Bookmark.edit with a blank url.
2017-07-13 22:58:02 -07:00
fcd507e337
Add new_photo_kwargs param to digest_directory.
2017-06-16 02:23:48 -07:00
707a9ab7c2
Fix a few bugs when photo.bytes is None.
2017-06-15 23:08:20 -07:00
6bed300496
Add more feature_required decorators and config.
2017-06-14 23:15:47 -07:00
a7cb922983
Fix missing exceptions import
2017-06-14 22:47:50 -07:00
bc5df9f1c2
Add max_len parameter to Tag.qualified_name.
2017-06-13 20:11:30 -07:00
8a0bbbcc56
Add /bookmarks/create_bookmark endpoint. No interface yet.
2017-06-12 20:15:02 -07:00
ac2b820a30
Add PDBTagMixin.get_root_tags
2017-06-11 22:57:08 -07:00
d10fe47dc4
Add parentheses to default tag chars.
2017-06-11 22:56:21 -07:00
821a5f3371
Move export functions to their own module.
2017-06-11 22:55:45 -07:00
4f03815a30
Floating point h Ⓜ️ s, and update to-do.
2017-06-11 21:09:17 -07:00
c88654346b
Fix not jsonifying the parent album.
2017-05-13 15:44:03 -07:00
4928095abb
Minor commentary and cleanup
2017-05-13 15:27:08 -07:00
55c14f3844
Don't leave bitrate undefined for non-temporal photos.
2017-05-12 17:50:31 -07:00
a4a278c173
Give Tags a description field.
2017-05-12 17:31:17 -07:00
a485be3c64
Make bitrate a real attribute instead of calculating in jinja
2017-05-11 22:34:19 -07:00
a5924b4642
I don't know how I forgot about dict(zip())
2017-05-11 22:32:55 -07:00
091dc82f1e
Catch expressionmatch.NoTokens for filename search
2017-05-06 18:37:02 -07:00
5b7c05c39d
Improve MMF search speed by pre-filtering photo IDs
...
Not perfected yet but it's an improvement.
Waste less time checking all photos, presearch for matching photos via photo_tag_rel table.
2017-05-01 21:27:54 -07:00
fbf73ac515
Since @transaction rolls back exceptions, don't use exception-based inserts.
2017-05-01 21:26:07 -07:00
a9c7ad6993
Improve separation between front & back with etiquette_flask package
...
Move flask-specific operations out of etiquette's files and into new etiquette_flask. In etiquette_site.py, etiquette calls are fully qualified.
2017-05-01 21:23:16 -07:00
83b9adbd61
Add more uncache points to help with stale data
2017-05-01 21:16:10 -07:00
8080d92f8d
add_associated_directory checks for dupe, so remove old check
2017-05-01 20:50:52 -07:00
06522ee372
Minor cleanup
2017-05-01 20:49:59 -07:00
43bc1e74fc
Add method Album.add_associated_directory
2017-05-01 20:43:45 -07:00
09f209719b
Add decorators.@required_feature to centralize FeatureDisabled exc.
2017-05-01 20:41:56 -07:00
af6785cead
Add decorators.@transaction to rollback sql upon exception
2017-05-01 20:23:58 -07:00
6ea3e67d86
Add method associated_directories.
2017-04-22 21:39:07 -07:00
528302b674
Relax Exists params; Add the .args tuple.
2017-04-22 21:20:03 -07:00
82ca1a08da
Add superinit to the objects.
2017-04-22 21:16:30 -07:00
d95eac4bb6
Minor improvements, update to-do list.
2017-04-20 18:31:48 -07:00
e3610eef2b
Standardize NotExclusive exception usage.
2017-04-20 18:29:30 -07:00
cc0c847eb5
Make sure easybake uses the normalized name in return notes.
2017-04-20 18:28:11 -07:00
24b6970617
Move default DB filename etc. to constants
2017-04-20 18:26:15 -07:00
241d1466eb
Fix tag expression bugs, move work to searchhelpers
...
Fix handling of bad tags, bad syntax, empty expression. Remove unnecessary warning_bag from matcher builder
2017-03-31 21:52:49 -07:00
791172e073
Support extension=* search, to filter extensionless files
2017-03-23 15:49:55 -07:00
5ee6062cad
Use None check instead of general falsey for minmax
2017-03-23 15:17:19 -07:00
