db28b6819c
Replace individual cache instantiations with all at once.
2018-05-06 21:08:56 -07:00
96d79f2b25
Oops, let separate loading of pragmas for not-first-time load.
...
Since DB_INIT was only run during first time setup, all future
loads were not running the pragmas, not even the FK restriction.
2018-05-03 19:02:53 -07:00
ae46b097e4
Let get_tag_by_name return inputted Tag right away.
...
For caller's convenience most functions assume you are passing
name as a string, but if you already have the Tag we'll just use it.
2018-05-03 18:20:48 -07:00
a8fc1206d9
Oops, move get_root_tags into alphabetical position.
2018-05-02 17:41:46 -07:00
1626a6fa11
Optimize PDB.get_root_albums and get_root_tags.
...
Achieves in a single query what used to require a ton of
individual get_parent calls.
2018-05-01 20:32:45 -07:00
fa97512586
Use batches of 999 for get_things_by_id.
...
SQLite has a limit on how many variables you can use per query.
2018-05-01 17:48:02 -07:00
e2cf9344c1
Copy the inputted new_photo_kwargs so as not to damage it.
...
Also I learned that pop takes a fallback.
2018-04-20 15:31:41 -07:00
fec5eaf21e
Add Tag.normalize_name to be called by PDB.normalize_tagname.
2018-04-20 15:28:27 -07:00
4434517c54
Rename config user.min_length -> min_username_length.
2018-04-15 14:36:07 -07:00
31b808bd64
Fix get_things_by_id not putting the results in cache.
2018-04-15 02:53:14 -07:00
0ee07023a1
Fix ids_needed being modified while iterating.
2018-04-15 02:35:46 -07:00
518a45ccd8
Add method get_things_by_id for mass lookups.
2018-04-15 02:14:06 -07:00
68d6e4faf4
Add explicit method get_album_by_id.
2018-04-15 02:10:42 -07:00
bfc4f313a9
Switch some formats to f-strings.
2018-04-15 01:13:02 -07:00
028a8cb2ef
All of the INTERSECTs can just be WHERE clauses.
2018-04-04 22:08:08 -07:00
c0df14db62
Fix typo binding -> bindings.
2018-04-04 21:40:39 -07:00
abe82dde9e
Let get_tag_by_name use cache after finding the ID.
2018-03-31 14:01:03 -07:00
d9f5765bcc
Use more self.sql_ instead of own cursors.
2018-03-29 19:24:48 -07:00
2edb9a1d57
Centralize cursors to PDB.sql_execute.
...
This gives me a nice common place to do logging if I want.
2018-03-29 19:24:39 -07:00
cca8837863
Move helpers.select_generator to PDB.sql_select.
2018-03-25 16:32:17 -07:00
da47b5bb3c
Fix several small, embarassing typos.
2018-03-23 01:18:43 -07:00
77dfeb4299
Add more _assert methods to hide clutter.
2018-03-23 00:35:56 -07:00
229530580f
Add Tag.normalize_description.
2018-03-23 00:35:56 -07:00
b4d3de3651
Add Bookmark.normalize_title and _url.
2018-03-23 00:35:56 -07:00
6ee86431e5
Add Album.normalize_title and _description.
2018-03-23 00:35:55 -07:00
1f7749b80a
PDB.new_album call album.add_photos instead of add_photo.
2018-03-22 23:11:25 -07:00
0e3ae11610
Improve normalizers, use less None; Add author search box.
...
It was getting difficult to remember which of the normalizers
use None and which don't. So let's try to be a little more consistent
and just use empty sets, etc, so the caller can rely on receiving a
set instead of having to check for None.
Also renamed search parameter authors->author to be more in line
with the singular form of extension.
2018-03-22 21:09:21 -07:00
088a79ffff
Add method PDB.get_users.
2018-03-22 21:06:05 -07:00
5be174d1b3
Completely rewrite search to use more SQL and less application.
...
In order to achieve tag_musts, we break each of the musts down
into separate EXISTS queries for each of the matchable children.
Then we INTERSECT those, and finally do other filtering and
ordering as usual.
2018-03-21 19:20:43 -07:00
1d76b4511b
Let PDB.get_tag convert normalization errors to NoSuch.
...
Instead of having to account for them on the other end,
since they don't care about that stuff anyway.
2018-03-21 11:03:58 -07:00
dffde094e8
Give Albums and Tags an author_id column.
...
Keeping V11 because I hadn't published previous commits yet.
2018-03-18 15:28:26 -07:00
d6d28764c4
Oops, assign self.config before dumping it.
2018-03-18 00:23:05 -07:00
c4d0de7f97
Move digest_directory and easybake to Util mixin.
2018-03-17 17:19:07 -07:00
87886a0a13
Unprivatize PDB._load_config and _save_config.
2018-03-17 17:07:30 -07:00
941d50eae0
Move config saving to own method.
2018-03-17 17:05:34 -07:00
44d1fb2fff
Add PhotoDB init option skip_version_check.
2018-03-13 02:25:04 -07:00
d85d911b22
Add PhotoDB init option create. Disable to require existing.
2018-03-13 02:24:11 -07:00
101f6d9c47
Add exception BadDataDirectory.
2018-03-13 02:21:35 -07:00
b89da5dec2
Add hasattr checks to prevent faulty __del__.
2018-03-13 02:20:17 -07:00
1cd78a678b
Minor linter appeasements.
2018-03-11 01:54:59 -08:00
94e518068f
Add Album.add_photos for batch adding with less savepoint waste.
...
Because each call to add_photo opens a savepoint, it is wasteful
to use it for a large number of photos that belong together.
2018-03-10 22:50:41 -08:00
f5a126d3bc
Oops, fix missing return statement.
2018-03-10 15:42:23 -08:00
42cef093a5
digest_directory: Use _normalize functions for argument prep.
...
This is a pattern I want to try out.
Not sure if it's ugly yet.
2018-03-10 15:32:31 -08:00
9355cd09ef
Add digest_directory parameter new_photo_ratelimit.
...
Initially inspired by the nuisance of having multiple photos
with the same timestamp because the system is too fast, this
feature could also be used to lessen disk load for running in
the background.
2018-03-10 14:18:28 -08:00
5f6d21fdee
Give Photos a searchhidden property.
...
By default, photos with searchhidden do not appear in the search
results. This allows a small number of representative images from
a large album to appear in the results, while the rest can be
found on the album's page.
The same effect could be achieved with a tag and forbid search,
but tag searching has much higher cost and it would be more difficult
to implement as a default behavior without requiring lots of special
checks whenever listing tags etc.
2018-03-09 17:10:27 -08:00
d88db08693
Allow PDB.rollback to take a savepoint; Warn early commits.
...
If a @transaction method makes calls to other @transaction
methods, and one of those nested calls makes a commit, and then
the outer call raises an exception, then the outer call will
not rollback properly because its savepoint is no longer in
the savepoint stack. So let's warn the user if that happens.
Should this raise an exception instead of just warn? Not sure,
I mean the data is already committed.
2018-03-05 21:35:36 -08:00
26141f8198
Put savepoints in the on_commit_queue so we can cancel on rb.
2018-03-02 20:30:55 -08:00
c001b0fc87
Use SQL savepoints to improve @transaction.
...
I didn't know about these! Now we don't have to roll all the way
back in case of problems. This allows for raising exceptions as
control flow since they won't undo unrelated stuff.
2018-02-25 17:10:04 -08:00
576fac416a
Move SQL heavy lifting to its own Mixin.
2018-02-25 16:30:05 -08:00
a2e13003e8
Add PhotoDB.sql_delete.
2018-02-25 16:09:36 -08:00
f34164bf85
Add search feature has_thumbnails.
...
Search needs a complete refactor.
But until then, let's keep adding to it!
2018-02-24 23:06:25 -08:00
13e8bc4a6f
Add more items to to-do list; minor fixes.
2018-02-20 16:50:39 -08:00
606a66cbf9
Remove Photo.filepath and real_filepath in favor of real_path.
2018-02-17 15:19:36 -08:00
9939f5da1d
For sql_insert / _update, let default commit=False.
...
Because all the calls are using it that way anyway.
2018-02-17 14:01:12 -08:00
2f9b51b636
Rename Photo.tags -> get_tags.
2018-02-16 23:07:21 -08:00
ac9d7ede22
Rename Album.photos -> get_photos.
2018-02-16 23:03:54 -08:00
fdcf07940a
Rename Groupable.children -> get_children.
2018-02-16 23:00:43 -08:00
75effafaf5
Rename Groupable.parent -> get_parent.
...
After making the "if x.parent is None" mistake too many times,
forgetting that it was not a @property, I've decided that the
verby name is better.
2018-02-16 22:59:34 -08:00
db92951ad6
Oops, remove incorrect name= from new_tag.
2018-02-16 22:57:35 -08:00
3c4f69f647
Specify name= on calls to PDB.get_tag.
...
Sometimes it's easy to forget whether its the name or ID,
and I like the consistency of specifying.
2018-02-16 22:25:56 -08:00
16ac6f8b85
Move UN / PW validation to helper _assert methods.
2018-02-16 22:18:19 -08:00
a8ead9c0c9
Replace a lot of manual insert/update with helper.
2018-02-16 20:19:18 -08:00
3e5031a1c5
Add methods sql_insert, sql_update to centralize sqlhelper work.
2018-02-16 18:49:02 -08:00
4e8a67aa54
Avoid sql indexing by instantiating User earlier.
2018-02-16 18:40:57 -08:00
bc4e228a25
Remove a lot of the commented code from earlier search.
2018-02-04 15:11:46 -08:00
f7dcb489c5
Remove unnecessary intification of timestamps.
...
I decided I want to have higher precision especially because
batch inserts were having several photos with the same
created timestamp making orderby >= dumb.
2018-02-03 01:35:32 -08:00
dec28b321a
Rename recursive_dict_update's parameters for clarity.
2018-02-03 01:10:54 -08:00
f266e1c79c
Split PhotoDB's large init into some smaller portions.
2018-02-03 01:10:04 -08:00
f9ee56187c
Rename attr database_file to database_filepath.
...
To make it more consistent with config_filepath and show
that it's a path object.
2018-01-30 19:46:43 -08:00
c62e6cd80b
Fix synonym fetcher using outdated column name.
2018-01-13 20:02:49 -08:00
14750eafc5
Move DB_INIT to constants; auto parse column names.
2018-01-13 16:48:48 -08:00
178a7df0b3
Add some more log debugs.
2018-01-11 19:32:15 -08:00
b5902ba4f1
Integrate voussoirkit sqlhelpers.
...
https://github.com/voussoir/else/tree/master/SQLHelpers
2018-01-09 21:15:50 -08:00
6f371701e4
Fix digest second-scan bug; Break down into smaller functions.
...
There was a bug where moving an album out of its determined parent
caused future digests to fail because I wanted to use the GroupExists
for control flow, but due to the @transaction decorator it was rolling
back the changes.
Moved some of the logical pieces of the function into subfunctions
as a visual and readability aid.
2018-01-09 18:59:15 -08:00
b859dbd22a
Add bookmark cache, replace get_bookmark code with call to get_thing.
2017-12-20 20:10:58 -08:00
0d0354f4da
Rename normalize_filepath to remove_path_badchars.
...
Because the function does not do any resolving or converting
to an absolute path, I feel the name 'normalize' is inaccurate
and could be misleading when read. This is more clear.
2017-12-07 21:15:10 -08:00
a7b5a7d42a
Multiple small cleanups, docstrings, messages, lint improvements.
2017-11-27 15:56:16 -08:00
f052858d5f
Improve log creation and use custom formatter.
2017-11-26 02:37:11 -08:00
64d8410977
No need for self.cur, just cur.
2017-11-16 18:03:51 -08:00
259c9ee1ab
Add PhotoDB init arg ephemeral. Uses :memory: sql and tempdir.
...
Primarily necessitated by unit testing. Running through the DB_INIT
is quite slow on disk, so this argument causes the sql to be done
on an in-memory database and all the other files are put into a
TemporaryDirectory.
Eventually I would like to have the other files be in-memory too
but that may be overcomplicated and underuseful.
2017-11-16 18:02:05 -08:00
8f000543ea
Add exceptions.DatabaseOutOfDate instead of just using an error string.
...
And don't raise SystemExit. Not sure what I was thinking.
2017-11-16 16:33:40 -08:00
009c0dc678
Allow the data_directory to contain colons (fully qualified paths)
2017-11-16 16:32:05 -08:00
a510c7b55c
Improve Album bytes caching, start caching photo count.
...
More careful uncaching of the summed bytes, to minimize
recalculation. Fewer cases where the album itself
is removed from the photodb's getter cache.
This also helps the download link on album pages disappear if
the child albums don't actually have any photos.
2017-11-12 15:21:53 -08:00
96856e9361
Cleanup: More minor dusting, commenting, clarity renaming.
2017-11-11 22:49:03 -08:00
bb5fa816d8
Cleanup: Rename GroupableMixin.add to add_child for clarity.
...
Especially since for albums it might sound like you're adding a photo.
2017-11-11 22:41:26 -08:00
3f69a2c240
Cleanup: Replace "objectid" with "object_id" in the Python portions.
2017-11-11 22:38:51 -08:00
a4b875b9ff
Fix config file not being updated when the default updates.
...
The len check was incorrect ever since I started nesting keys.
Now we measure all of the keys recursively.
2017-10-05 18:39:34 -07:00
0849796986
Make build_query params all optional, more kwargy.
2017-09-23 11:22:05 -07:00
163e960bfa
Fix unrecognized orderby not using the warning bag.
2017-09-23 11:08:19 -07:00
557f78eb02
Integrate author search into the query instead of postprocessing.
...
Much faster this way.
2017-09-23 11:04:26 -07:00
38e72c0c3a
Make notnulls set instead of list.
2017-09-23 11:00:37 -07:00
89d9c1d893
Move the new photo creation out of the exception handler.
...
I don't like the "during the handling of the above exception" because
it's misleading as to the source of the problem.
2017-08-01 18:13:54 -07:00
bd847159fa
Organize config by nesting related stuff.
...
I thought about switching to .ini files but that's even less desirable.
2017-07-29 16:23:15 -07:00
4fc203634c
Add new config "enable_login".
2017-07-20 23:25:23 -07:00
fcd507e337
Add new_photo_kwargs param to digest_directory.
2017-06-16 02:23:48 -07:00
ac2b820a30
Add PDBTagMixin.get_root_tags
2017-06-11 22:57:08 -07:00
821a5f3371
Move export functions to their own module.
2017-06-11 22:55:45 -07:00
4928095abb
Minor commentary and cleanup
2017-05-13 15:27:08 -07:00
a4a278c173
Give Tags a description field.
2017-05-12 17:31:17 -07:00
091dc82f1e
Catch expressionmatch.NoTokens for filename search
2017-05-06 18:37:02 -07:00
