Commit graph

619 commits

Author SHA1 Message Date
37bff5c427 Look for renamed files (lost&found) during digest_directory. 2020-09-15 14:57:05 -07:00
9578dc4d67 Add column for photo's st_dev, st_ino to facilitate lost&found. 2020-09-15 14:56:45 -07:00
50ecc4c1c1 Let digest_directory act as a generator, yielding photos & albums. 2020-09-15 13:48:06 -07:00
37c784982a Add PDB.clear_all_caches. 2020-09-15 13:44:49 -07:00
5bc3fdd08e Move purge_empty_albums to PDBAlbumMixin. 2020-09-15 13:44:28 -07:00
5b4cc5bf9c Log the exception before rolling back. 2020-09-14 20:25:24 -07:00
5d40622b76 Add log for Photo.relocate. 2020-09-14 19:38:29 -07:00
be0747fd8c Use path.absolute_path when logging add_associated_directory. 2020-09-14 19:38:13 -07:00
8279a1871b Return set instead of list from get_parents, get_tags. 2020-09-14 18:34:14 -07:00
a32caafb8b Remove leftover print statements. 2020-09-14 16:56:55 -07:00
1f7ffbb283 Fix old reference to incorrect variable name tagname. 2020-09-14 05:50:10 -07:00
f3bceb4688 Remove get_cached_flat_dict, add get_cached_tag_export. 2020-09-14 05:49:24 -07:00
cf64c79809 When grouping tags, update tagged photos to prefer child.
I am also considering applying the opposite effect when ungrouping.
Should a photo with A.B get A when A and B are ungrouped? There are
some cases where the answer is yes, and others no, depending on the
reason you're ungrouping the tags. Whereas this change is non-
controversial and simply enforces the existing standard of adding
more specific tags to a photo.
2020-09-14 05:46:43 -07:00
f1d17350a7 Add Photo.remove_tags. 2020-09-14 05:38:20 -07:00
85f24dd1f5 Let Photo.remove_tag only remove exact tag, not children.
I wrote this because I felt it would be a useful shorthand, as a way
of nuking all tags of a subtree off a photo, but it's too easy
to cause collateral damage when composing remove_tag with other
functions. So, when you remove tags from a photo, you'll have to be
more specific.
2020-09-14 05:37:47 -07:00
54add8ad61 Add search parameter yield_albums.
The exact details of this feature are still experimental, but I think
the concept is sound enough to make it an official parameter.
2020-09-14 05:16:36 -07:00
188095c30b Remove unnecessary intermediate variable syn_lines. 2020-09-12 17:33:45 -07:00
5dcccae646 Fix add_children all(bail) causing short-circuit. 2020-09-12 13:14:40 -07:00
d41bad63e7 Move reset of cached_tag_flat_dict out of GroupableMixin into Tag.
That should not have been there!
Needed to add the BAIL sentinel so that Tag methods would only
reset the cache if the internal method actually did made any changes.
2020-09-12 01:19:03 -07:00
74445d0a1f Add internal _add_child so that plural doesn't call transacted.
add_children was calling add_child in a loop. Since this is a parent
class, that call would actually call the subclasses' add_child method,
which has a transaction decorator, and create unnecessary transactions.
2020-09-12 01:00:39 -07:00
21921dc8fc Rename cached_frozen_children -> cached_tag_flat_dict. 2020-09-12 00:56:32 -07:00
b59c2700db Update _lift_children docstring. 2020-09-12 00:53:12 -07:00
1a53af8821 Replace horrible flat_dict algorithm with actual recursion. 2020-09-12 00:21:37 -07:00
2a9b2cd095 Add methods PDB.get_thing_count for all thing types. 2020-09-11 23:51:57 -07:00
cfb48a8d11 Add disclaimer about yield order to get_things_by_id. 2020-09-11 22:13:30 -07:00
0cdd8c0d71 Add detailed doctstring to digest_directory. 2020-09-11 15:49:02 -07:00
4a3236babe Update digest_directory to handle plural albums at every step. 2020-09-11 15:48:37 -07:00
dea746c45a Alphabetize Album's methods. 2020-09-11 15:21:13 -07:00
1cc70616a5 Replace get_album_by_path with get_albums_by_path generator. 2020-09-11 15:15:12 -07:00
7705848714 Let new_album take multiple associated directories. 2020-09-11 15:02:33 -07:00
519bcf32e6 Add separate plural method Album.add_associated_directories. 2020-09-11 15:02:15 -07:00
8ed5f0be51 Fix tag_expression usage of frozen_children which was tag objects.
At some point, I changed tag_export.flat_dict from using strings as
keys to using actual Tag objects as keys. The tag expression handler
never got updated, so frozen_children[tagname] raised KeyError as
it was looking for a string.
I considered using tag_expression_tree.map to convert all the tree
tokens into tag objects, but when we render the tree back into text
it will say "Tag:name" instead of just "name" throughout the whole
expression, and I don't want to deal with converting those back.
2020-09-09 12:58:57 -07:00
30d7dbdb39 Fix tag_export.flat_dict which was not truthful about synonyms. 2020-09-09 12:53:51 -07:00
a79e14e120 Let search_kwargs[orderby] give back None instead of empty list. 2020-09-09 12:20:49 -07:00
7eb095e17c Experiment: Use width*height instead of area column when searching.
Will see if this has any adverse effects, or if it goes well I can
remove the area and ratio columns.
2020-09-08 21:59:42 -07:00
a8fa621f01 Rewrite helpers.comma_space_split with a simple regex. 2020-09-04 17:00:01 -07:00
e774b5f101 Fix zip_photos attempting to add deleted files. 2020-08-30 18:48:08 -07:00
0b4542089b Accommodate chunk_sequence which now always yields lists. 2020-08-28 17:57:15 -07:00
e40c91a7cf Split reload_metadata into separate functions by mimetype. 2020-08-28 17:51:07 -07:00
45d8809971 Fix crash while trying to get size of deleted file.
Pathclass no longer returns None for missing files, as it did when
this was written.
2020-08-28 17:32:53 -07:00
fa8bbf92f9 Remove call to logging.basicconfig, let the frontend do it. 2020-08-12 12:23:51 -07:00
b15beb73eb Rewrite chunk_sequence to work with generators. 2020-08-12 11:21:48 -07:00
b21e6a1206 Show data directory in DatabaseOutOfDate message, not db file. 2020-07-01 16:19:54 -07:00
363d0bac67 Move THING_CLASSES and thing-related methods to new mixin.
I realized there's no need to have that on the global scope when
it is only used for the thing-methods.
2020-04-15 16:33:16 -07:00
5dffe52830 Start _cached_sql_tables as None, and in the SQL mixin. 2020-04-15 16:31:47 -07:00
35895856e6 Use .clear() instead of reassigning to empty lists.
I cannot remember if there is a reason I chose to use empty lists.
If there is, I'll rediscover it and write a comment. Until then, it
makes sense to use clear as an explicit clearing of the list.
2020-04-15 16:31:09 -07:00
43a665fd91 Replace more %s with fstrings. 2020-04-15 16:29:56 -07:00
930960e22a Move this config prep code into voussoirkit as configlayers. 2020-04-03 17:09:01 -07:00
9c96522cfc Remove thing_map table, just use thing_map class.table. 2020-04-03 15:20:09 -07:00
545ea62818 Add some docstrings and comments near get_cached_instance. 2020-04-03 12:33:19 -07:00
5a16579fe3 Add a 'type' string to all etiquette jsonify outputs. 2020-04-02 22:43:20 -07:00
dd3d40de53 Experimental: Allow search results to include albums.
I've been thinking about this for a while but couldn't think of
the perfect way to implement it. I still haven't, so instead I'm
just starting with something and we'll see how to improve later.
At any rate, I can update the rest of the system to expect Albums
coming out of search so that if I ever have a better algorithm
everything else will already be ready for it.
For this first experiment, just any photos that are part of an album
will send that album out as a result. It doesn't even respect the
limit parameter, it's really just to see how it feels to use.
2020-04-02 22:36:40 -07:00
e94e35c98c Add option to natural_sort filenames before digesting. 2020-04-02 21:57:54 -07:00
374763f990 Reduce log level of savepoints to 5.
So far I am satisfied with how they are working, and they clutter
the output so much I can't read anything else.
For reference, logging.DEBUG is 10.
2020-04-02 21:56:41 -07:00
0fdf08adf5 Fix bug where searching musts=[A, A.B] ignored B.
The problem was that during the expansion of A, B would get added to
the `total` set. And then because B was already in total, it would
bail early instead of expanding, but that meant it got left out of
the final query. But this only happened sometimes since random set
ordering would sometimes expand B before A and everything was ok.
I think the best solution is to do this first pass that eliminates
ancestors. Furthermore, I realized that the entirety of expand_nested
hinged on this flawed logic and was otherwise completely redundant
for a simple list comprehension.
2020-03-19 19:04:09 -07:00
13c3a1c0e9 Update docstrings for walk_children, walk_parents. 2020-03-19 19:03:47 -07:00
0a8a754cbc Replace set([]) with {}. 2020-03-19 17:40:52 -07:00
e35efb5c61 Make this string an r-string so it gets better syntax highlighting. 2020-03-11 18:11:54 -07:00
6529aedde6 Add docstring to PDB.__init__. 2020-03-06 09:43:16 -08:00
3f2cde02e1 Use send2trash to support recycle_instead_of_delete. 2020-02-27 14:18:46 -08:00
961ece0081 digest_directory excludes extend defaults instead of overriding them.
With this change, there is currently no way to specify that I actually
want to include thumbs.db etc, but I think the likelikhood of wanting
to extend the defaults greatly exceeds wanting to overrid them.
2020-02-27 14:17:30 -08:00
9d620b4b97 Remove unused imports & other small things. 2020-02-20 22:44:56 -08:00
854fa4db51 Rename PDB.register_user -> new_user.
I was treating User a little bit special here, but I prefer to
have the more consistent terminology.
2020-02-20 00:18:40 -08:00
1fc2ed09f5 Remove commit argument from ALL methods. @transaction does it. 2020-02-19 23:56:09 -08:00
0a984117c5 Add rollback queue, like commit queue. 2020-02-19 23:46:30 -08:00
8c854f3b6a Improve this comment about Photo rename logic. 2020-02-19 23:22:08 -08:00
f99d3d03d9 Photo.rename new_path.assert_not_exists. 2020-02-19 23:12:29 -08:00
00a443eec3 Always use the commit queue, don't bypass with commit=True. 2020-02-19 23:12:11 -08:00
7f3602d673 Remove @transaction from mixin methods, apply it to object methods. 2020-02-19 23:10:49 -08:00
953a85c998 Delete method Tag.leave_group, leftover from 1-parent days. 2020-02-19 23:05:55 -08:00
8c797024e0 Remove calls to pdb.commit(). Let @transaction do it. 2020-02-19 22:58:21 -08:00
203fb9e00f @transaction intercept commit argument; commit / release here. 2020-02-19 22:36:00 -08:00
04f3f6f297 Remove commit=False from all method calls, it's default now. 2020-02-19 22:20:21 -08:00
3a9d7fc2dc Set default commit=False for ALL methods.
I'm working on tightening up some of the transaction code. In the
past it was default commit=True because I would launch the repl, do
something, and quit, so it was nice to have auto commit. But really
it makes more sense to have it default False and be explicit when
to commit.
2020-02-19 20:08:24 -08:00
108ba8bd95 Add PDB.release_savepoint. 2020-02-19 17:00:08 -08:00
bcd13b9ae4 Make the commit queue task popping a little more clear. 2020-02-19 16:26:23 -08:00
e7c90faf79 PhotoDB.rollback without savepoint should roll all, like real sql.
In sqlite3, rollback without a savepoint undoes the entire pending
transaction stack, whereas I was just popping the last save.
This change makes it match real sqlite. Also the only current call
for rollback is in the @transaction decorator which is already
explicit.
2020-02-19 16:26:02 -08:00
9e0051ca9c Photo.move_file assert new path doesn't exist yet. 2020-02-19 16:23:31 -08:00
1375d9fb26 Fix new_photo, data[extension] to use new pathclass.Extension. 2020-02-19 15:11:52 -08:00
5da8d793d6 Return this None instead of assigning then returning. 2020-02-19 14:06:53 -08:00
31517a41d2 Add missing period on exception message. 2020-02-19 13:11:57 -08:00
92f3569971 Use of extract_table_column_map has moved to sqlhelpers. 2020-02-07 15:15:49 -08:00
b88d577ddd Use voussoirkit.imagetools for fit_into_bounds function. 2020-02-04 19:35:46 -08:00
25ed427272 Wrap DB_INIT in a transaction. 2020-02-04 18:43:30 -08:00
d6d7521bce Assert table exists for any sql op involving argument tables.
At the moment, all of these functions are safe because they're
called with hardcoded tables determined by other code, not user input.
But while I was working in this area I felt it would be good to add
a safety check just in case.
2020-02-04 18:15:14 -08:00
64f9eb5f2b Replace some % formatting with fstrings. 2020-02-04 18:12:41 -08:00
eff0efdf40 Indicate required feature in exception message. 2020-01-29 23:52:25 -08:00
2ffe3940e6 Update OutOfDate message to show actual argument. 2020-01-28 10:52:19 -08:00
93eb693bee Separate version_pragma from other pragmas!
This was a dumb bug. Because the version was included in the pragmas
that get set on every load, the database was receiving the new
user_version simply by setting skip_version_check=False and letting
the regular pragmas load, so all future checks passed without
having to run the db upgrader.
2020-01-27 20:46:32 -08:00
222d63857e Replace which with winwhich. 2020-01-21 23:50:11 -08:00
1b308d7921 Fix seconds_to_hms bug!
Embarrassing. If M was zero then you got HH:SS.
2020-01-06 20:21:41 -08:00
3bd8fe6392 Raise FileNotFound from read_filebytes. 2019-12-10 12:48:28 -08:00
9fa49bbddf Add parameter only_shrink to fit_into_bounds. 2019-08-14 13:48:22 -07:00
947d05a4fa Revert 488d793, reinstate PhotoDB.normalize_tagname.
In that commit, I missed three more calls in objects.py and
now I'm deciding that 5x copy-paste is not worth it.
2019-06-15 16:49:52 -07:00
bdff14ecf2 Identify a logic error without actually fixing it.
😎
2019-04-26 23:14:14 -07:00
cc6e7a8b16 Improve line consistency across the PhotoDB.new_* methods.
By clarifying why some lines are above / below the generate_id
call and putting similar code in similar order to each other.
2019-04-26 23:12:55 -07:00
a9865d8546 Replace PhotoDB.normalize_tagname with real call to Tag.norm_name.
I originally did this because I didn't want to accidentally call
Tag.normalize_name and forget to pass the valid parameters. However,
having this single method be on PhotoDB while the other norms are
part of their proper class has been an eyesore.
So since there are only a few calls to this I'm just inlining them
and trusting to not forget if I add more in the future.
2019-04-26 23:06:41 -07:00
8c74e42a74 Add Album.has_any_associated_directory. 2019-04-26 18:55:03 -07:00
ebde5c7ee8 Give each object class a table and centralize db_row normer. 2019-04-26 18:52:56 -07:00
fa97dd5d3c Remove Tag._cached_synonyms. Not worthwhile. 2019-04-05 23:26:11 -07:00
d24f93809b Add message parameter to commit instead of logging separately.
The upside is that we can get rid of some redundancy and reduce
the friction of adding more commit messages.
The downside of this is that the log statement always reports from
commit, instead of the function calling commit. But with unique
messages this shouldn't be too much trouble and should be worth it.
2019-04-01 23:29:01 -07:00
68a567d266 Perform is None check before detailed type checks. 2019-04-01 23:17:35 -07:00
3aa064cde3 Move log statement to top of method. 2019-04-01 23:17:01 -07:00
2f33023025 Replace singular guard continue with simple if. 2019-04-01 23:16:25 -07:00
ad140e485e Two small docstring improvements. 2019-03-16 13:09:02 -07:00
d66dc53215 Require voussoirkit 0.0.28 for sqlhelpers.listify.
Instead of defining that within etiquette.
2019-01-01 18:08:47 -08:00
5d7bc1ff4b Somewhat improve prettiness of hyphen range parse code. 2018-12-27 15:31:56 -08:00
c6c6158062 Save this assignment until it's needed. 2018-12-21 23:44:50 -08:00
7674610b88 Somewhat simplify a few conditions in photodb init. 2018-11-24 22:25:25 -08:00
1eb5a6f465 Rename DatabaseOutOfDate's parameter 'current' -> 'existing'. 2018-11-24 22:06:16 -08:00
707fdcc637 Add album cards and improve album pages.
- album card has placeholder for future thumbnail.
- replaced nested tree hierarchy lists with separate boxes.
- list/grid view also applies to the root listing.
- added a sticky right panel for all the tools. not pretty yet.
- mechanism for adding sticky panel changed. instead of applying
  it to the #right, you apply it to #content_body so that its
  grid layout can be updated properly.
2018-11-12 22:15:59 -08:00
d0208154e4 Add a function to clarify SQL_COLUMNS creation. 2018-11-11 19:08:31 -08:00
dbdd509cd2 Rearrange imports so voussoirkit is right after lib imports.
Added a styleguide.md file to refer back to.
Since voussoirkit is a library it feels better to have it below
the rest of the library and above the local project imports.
2018-11-04 19:27:20 -08:00
de48105e32 When thumbnailing, don't call relative_to on return.
The path given by make_thumbnail_filepath will already have
it in the right place, so this is probably an accident waiting
to happen.
2018-11-04 13:30:34 -08:00
25a27e6241 Move dict_to_params to flask-specific helpers.py.
The rest of etiquette has nothing to do with URLs so this
doesn't belong in the backend helper file.
2018-11-04 13:30:08 -08:00
20e41ad689 Extremely minor touchups. 2018-09-23 14:57:25 -07:00
95edc4d397 Add method Photo.move_file. 2018-09-23 14:54:56 -07:00
4d1575719b Bump voussoirkit requirement to 0.0.24 for pathclass exceptions. 2018-09-23 14:54:11 -07:00
e9f567ffc9 Don't allow Groupable to add itself as a child.
omg what an oversight
2018-09-22 23:14:41 -07:00
670bc8b375 Allow setting searchhidden during new_photo. 2018-09-21 21:31:57 -07:00
0ccfa74709 Let helpers.generate_*_thumbnail raise FileNotFoundError. 2018-08-17 22:05:47 -07:00
bea9f905bd Support downloading .zip of arbitrary photos, clipboard.
Now that creating zips of any photo set is easier, we can
let the user download whatever is on their clipboard.
2018-08-14 23:02:06 -07:00
bc6a0aa907 Improve & generalize zipfile code.
Moved some heavy lifting out of the flask album.zip endpoint
and into helpers.py.
Renamed some things for clarity.
2018-08-14 22:58:26 -07:00
53c86c30a1 Improve clarity of the recursive_dict_ helper functions & usage. 2018-08-11 18:19:55 -07:00
3b3c690ae3 Remove return value from load_config, so no illusion of purity.
This method already relies on object state anyway and writes back
to self.config, so the return value is unnecessary.
2018-08-11 18:19:10 -07:00
9eb9220ca9 Read and write the config file as UTF-8. 2018-08-11 17:57:19 -07:00
fb052a2d56 Fix Album.add_associated_directory not letting duplicates.
The docstring and the actual code were disagreeing.
2018-07-29 16:30:30 -07:00
30e3aa9c6f Simplify and improve rollback logic. 2018-07-29 16:28:57 -07:00
30d96139c2 Remove commit param from PDB.sql_* methods nobody was using.
And is better left to the caller anyway.
2018-07-29 16:05:49 -07:00
9328600355 Add message parameter to PDB.savepoint for better logging.
Helps to have a visual on which methods are creating the saves.
2018-07-29 16:03:04 -07:00
56e7472c93 Revert f-strings to %s for logging calls.
In my excitement I forgot about that rule.
2018-07-29 16:01:26 -07:00
8447fb3343 Make deleting and unlinking tags require confirmation.
Added text inside the delete buttons and made them confirm.
2018-07-28 20:25:25 -07:00
1b90850818 Oops, add the required_feature decorators to remove_child. 2018-07-22 21:06:16 -07:00
2a45f4e17e Add parameter albums to purge_empty_albums as starting set.
Like purge_deleted_files, you can provide a list of albums to
act as the starting point instead of letting it use the default
of all albums.
2018-07-22 20:21:00 -07:00
8a12a24e8e Add tag_autocomplete.js.
Instead of embedding the entire tag list in the search.html template
every single time, this script loads the tags from the new,
cache-enabled endpoint /all_tags.json. Then we can use html5
datalists to create autocomplete forms on the search and photo pages.
2018-07-22 20:19:05 -07:00
bbf07f4401 Fix image thumbnail wrong variable bug.
How long has that been there?
2018-07-22 20:19:05 -07:00
d740e6d686 Oops, fix read_filebytes not stopping at the range_max properly. 2018-07-22 20:18:58 -07:00
4c65ccaf68 Big! Liberate Groupables from strict heirarchy. Multiple parents.
I found that the strict heirarchy was not satisfying the situation
where one tag is the intersection of two others, but we can only
pick one as the parent

For example, does red_jacket belong under clothes.red_clothes or
clothes.jackets? A search for "red_clothes AND jackets" might
give us someone wearing red pants and a black jacket, so this
definitely needs to be a separate tag, but picking only one
parent for it is not sufficient. Now, a search for red_clothes
and a search for jackets will both find our red_jacket photo.

The change also applies to Albums because why not, and I'm sure
a similar case can be made.

Unfortunately this means tags no longer have one true qualname.
The concept of qualnames has not been completely phased out but
it's in progress.

This commit is very big because I was not sure for a long time
whether to go through with it, and so much stuff had to change
that I don't want to go back and figure out what could be grouped
together.
2018-07-20 13:09:06 -07:00
4987db4bd0 Switch to f-string formatting in many places. 2018-07-18 18:36:36 -07:00
4a39873fed Increase default tag cache to 10,000. 2018-07-18 18:27:18 -07:00
38bac41eb9 Add helpers.split_easybake_string for prelim parsing.
Helps to remove some distractions from the main easybake method.
2018-07-15 14:23:43 -07:00
f06b0915ab Add new_photo_kwargs to the create_or_get call.
Due to the closure, it was already working just fine, but I
want the explicitness of having it there.
2018-07-15 13:52:08 -07:00
57a9e0d4dc Move PDB.close out of the sqlmixin, into main class.
Doesn't really deserve to be with the sql stuff.
2018-07-15 13:27:04 -07:00
c37b54b456 Add dividers and rearrange some of the table column parsing. 2018-07-13 19:22:05 -07:00
0852630cd1 Alphabetize _extract_table_name. 2018-06-30 12:55:30 -07:00
c930228d3f Rearrange order of Album.delete ops to not void FK contraint. 2018-06-30 12:51:09 -07:00
c736c33165 Add caching of tag synonyms. 2018-05-06 21:09:15 -07:00
c6b71f1a0d Revert caching of qualified names. Causing too many problems.
The world is just not ready for it. Was having issues about the
parents being deleted / renamed and needing to propogate those
changes in a not-ugly way.
Will reassess in the future.
2018-05-06 21:09:09 -07:00
0c8a57b6aa Make the assert methods public. 2018-05-06 21:09:09 -07:00
40c255b0d0 Pass object instantiations through a cacher.
I want the system to only have one instance of a particular
object at any time, so that we can effectively cache things
in them.
2018-05-06 21:09:08 -07:00
11fda94968 Make the SQL_COLUMNS code a bit easier to read. 2018-05-06 21:09:02 -07:00
db28b6819c Replace individual cache instantiations with all at once. 2018-05-06 21:08:56 -07:00
96d79f2b25 Oops, let separate loading of pragmas for not-first-time load.
Since DB_INIT was only run during first time setup, all future
loads were not running the pragmas, not even the FK restriction.
2018-05-03 19:02:53 -07:00
64449569ea Rearrange statements during tag delete to not voilate FK. 2018-05-03 18:59:50 -07:00
23cc769357 Take advantage of parent's cached_qualname when making mine. 2018-05-03 18:45:57 -07:00
943deb9bf7 Move child-lifting code to its own helper method. 2018-05-03 18:32:44 -07:00
ae46b097e4 Let get_tag_by_name return inputted Tag right away.
For caller's convenience most functions assume you are passing
name as a string, but if you already have the Tag we'll just use it.
2018-05-03 18:20:48 -07:00
a0e3e037c1 Bail out of join_group early if already immediate parent.
I'd rather make the extra select than a delete and reinsert.
2018-05-03 18:18:13 -07:00
22269af2ca Always use False for leave commit before joining new group. 2018-05-03 18:10:12 -07:00
a8fc1206d9 Oops, move get_root_tags into alphabetical position. 2018-05-02 17:41:46 -07:00
58d450a856 Fix handling of the event that a file loses its thumbnail.
In which case return_filepath is None and the relative_to breaks.
2018-05-01 20:41:38 -07:00
1626a6fa11 Optimize PDB.get_root_albums and get_root_tags.
Achieves in a single query what used to require a ton of
individual get_parent calls.
2018-05-01 20:32:45 -07:00
48396d6ab6 Add Album.remove_photos for batch removals. 2018-05-01 18:05:07 -07:00
698981dd29 Move _uncache_sums call out of _add_photo, into callers. 2018-05-01 17:48:56 -07:00
fa97512586 Use batches of 999 for get_things_by_id.
SQLite has a limit on how many variables you can use per query.
2018-05-01 17:48:02 -07:00
97e1f881d0 Move video thumbnailing code to helpers. 2018-04-28 20:36:33 -07:00
5bc2bbdacb Move image thumbnailing code to helpers. 2018-04-28 20:33:05 -07:00
41823394eb Use PDB.get_albums_by_id for Photo.get_albums. 2018-04-28 19:17:11 -07:00
62496d183e Use PDB.get_tags_by_id for Photo.get_tags. 2018-04-28 19:14:58 -07:00
b19682d1a5 Replace Photo.sorted_tags with jinja filter. 2018-04-28 19:05:51 -07:00
df7fbeae0a Minor clarity improvements to required_feature logic. 2018-04-27 20:04:18 -07:00
e2cf9344c1 Copy the inputted new_photo_kwargs so as not to damage it.
Also I learned that pop takes a fallback.
2018-04-20 15:31:41 -07:00
fec5eaf21e Add Tag.normalize_name to be called by PDB.normalize_tagname. 2018-04-20 15:28:27 -07:00
4434517c54 Rename config user.min_length -> min_username_length. 2018-04-15 14:36:07 -07:00
21bd211889 Give Users a display_name. 2018-04-15 14:23:24 -07:00
8562b355ce Fix typo in debug commit message. 2018-04-15 13:49:10 -07:00
31b808bd64 Fix get_things_by_id not putting the results in cache. 2018-04-15 02:53:14 -07:00
7d8b8daeba Rewrite Album.sum_bytes to use SQL sum in a single query. 2018-04-15 02:52:41 -07:00
f77ab51b14 Add Album.has_any_photo and has_any_subalbum_photo. 2018-04-15 02:41:24 -07:00
0ee07023a1 Fix ids_needed being modified while iterating. 2018-04-15 02:35:46 -07:00
518a45ccd8 Add method get_things_by_id for mass lookups. 2018-04-15 02:14:06 -07:00
68d6e4faf4 Add explicit method get_album_by_id. 2018-04-15 02:10:42 -07:00
0094e7c006 Fix bug where add_tag supersedence caused early commit. 2018-04-15 01:14:20 -07:00
bfc4f313a9 Switch some formats to f-strings. 2018-04-15 01:13:02 -07:00
028a8cb2ef All of the INTERSECTs can just be WHERE clauses. 2018-04-04 22:08:08 -07:00
c0df14db62 Fix typo binding -> bindings. 2018-04-04 21:40:39 -07:00
abe82dde9e Let get_tag_by_name use cache after finding the ID. 2018-03-31 14:01:03 -07:00
d9f5765bcc Use more self.sql_ instead of own cursors. 2018-03-29 19:24:48 -07:00
144e97d365 Use self.photodb.sql_ methods instead of running own cursors.
All in the name of centralization.
Also improved SQLness of Tag.convert_to_synonym.
2018-03-29 19:24:48 -07:00
2edb9a1d57 Centralize cursors to PDB.sql_execute.
This gives me a nice common place to do logging if I want.
2018-03-29 19:24:39 -07:00
cca8837863 Move helpers.select_generator to PDB.sql_select. 2018-03-25 16:32:17 -07:00
da47b5bb3c Fix several small, embarassing typos. 2018-03-23 01:18:43 -07:00
77dfeb4299 Add more _assert methods to hide clutter. 2018-03-23 00:35:56 -07:00
229530580f Add Tag.normalize_description. 2018-03-23 00:35:56 -07:00
b4d3de3651 Add Bookmark.normalize_title and _url. 2018-03-23 00:35:56 -07:00
6ee86431e5 Add Album.normalize_title and _description. 2018-03-23 00:35:55 -07:00
dddd8a3aa1 Add ObjectBase.normalize_author_id. 2018-03-23 00:35:47 -07:00
1f7749b80a PDB.new_album call album.add_photos instead of add_photo. 2018-03-22 23:11:25 -07:00
0e3ae11610 Improve normalizers, use less None; Add author search box.
It was getting difficult to remember which of the normalizers
use None and which don't. So let's try to be a little more consistent
and just use empty sets, etc, so the caller can rely on receiving a
set instead of having to check for None.

Also renamed search parameter authors->author to be more in line
with the singular form of extension.
2018-03-22 21:09:21 -07:00
088a79ffff Add method PDB.get_users. 2018-03-22 21:06:05 -07:00
5c89e13129 Update DatabaseOutOfDate message to use new upgrader path. 2018-03-22 17:34:43 -07:00
5be174d1b3 Completely rewrite search to use more SQL and less application.
In order to achieve tag_musts, we break each of the musts down
into separate EXISTS queries for each of the matchable children.
Then we INTERSECT those, and finally do other filtering and
ordering as usual.
2018-03-21 19:20:43 -07:00
1e42f1ea4e DB V12: Add (photoid, tagid) index to photo_tag_rel. 2018-03-21 19:00:13 -07:00
acc9b67a20 Add parameter minimal to jsonify.tag. 2018-03-21 18:48:38 -07:00
6c77ddd1c3 searchhelpers.normalize_author use un only, not IDs.
I don't like the idea of a user setting their username to be
someone else's ID.
2018-03-21 13:21:25 -07:00
afc5f38a8d Combine normalize_limit, _offset to normalize_positive_integer. 2018-03-21 11:25:21 -07:00
1d76b4511b Let PDB.get_tag convert normalization errors to NoSuch.
Instead of having to account for them on the other end,
since they don't care about that stuff anyway.
2018-03-21 11:03:58 -07:00
9a1caee412 Add helper function hash_file. 2018-03-18 20:56:29 -07:00
2d73e59abf Let helpers.read_filebytes operate on pathclass Paths. 2018-03-18 20:56:08 -07:00
e186653e36 Use shutil.which to discover ffmpeg.
How did I miss such an easy solution for so long??
So glad I did not go the env var route.
2018-03-18 17:03:11 -07:00
84e0266f30 Add author to object jsonifiers. 2018-03-18 15:59:36 -07:00
60049c777f Add method get_author to all objects. 2018-03-18 15:58:51 -07:00
dffde094e8 Give Albums and Tags an author_id column.
Keeping V11 because I hadn't published previous commits yet.
2018-03-18 15:28:26 -07:00
d6d28764c4 Oops, assign self.config before dumping it. 2018-03-18 00:23:05 -07:00
6add02c8a2 Move column extractor to own function. 2018-03-18 00:21:03 -07:00
d383000441 Add Foreign Keys, Primary Keys, and NOT NULL constraints. 2018-03-18 00:09:08 -07:00
33d506ddaf Rearrange tables in preparation for foreign keys.
In order to create a foreign key the table has to already exist.
So move the object types to the top, with Users first since other
things have authors.
2018-03-17 17:54:15 -07:00
c4d0de7f97 Move digest_directory and easybake to Util mixin. 2018-03-17 17:19:07 -07:00
87886a0a13 Unprivatize PDB._load_config and _save_config. 2018-03-17 17:07:30 -07:00
941d50eae0 Move config saving to own method. 2018-03-17 17:05:34 -07:00
269cfb6762 Improve some variable names for clarity. 2018-03-17 16:48:09 -07:00
ffe0be21f5 Fix Photo thumbnail paths to be relative to the PDB's thumb dir.
Previously they were relative to the directory from which the client
was launched, meaning it usually started with ./_etiquette.
Now it should be more portable and less tight.
2018-03-13 02:50:54 -07:00
44d1fb2fff Add PhotoDB init option skip_version_check. 2018-03-13 02:25:04 -07:00
d85d911b22 Add PhotoDB init option create. Disable to require existing. 2018-03-13 02:24:11 -07:00
101f6d9c47 Add exception BadDataDirectory. 2018-03-13 02:21:35 -07:00
b89da5dec2 Add hasattr checks to prevent faulty __del__. 2018-03-13 02:20:17 -07:00
1cd78a678b Minor linter appeasements. 2018-03-11 01:54:59 -08:00
c99e783d04 Rename comprehension variable to not look like a conflict.
The existing code worked fine, but it was odd to see that `tag`
was both the loop variable as well as the key into the children.
So let's change it.
2018-03-11 01:35:35 -08:00
a86c27ceb4 Simplify some unnecessarily detailed log formats. 2018-03-11 01:34:32 -08:00
94e518068f Add Album.add_photos for batch adding with less savepoint waste.
Because each call to add_photo opens a savepoint, it is wasteful
to use it for a large number of photos that belong together.
2018-03-10 22:50:41 -08:00
f5a126d3bc Oops, fix missing return statement. 2018-03-10 15:42:23 -08:00
42cef093a5 digest_directory: Use _normalize functions for argument prep.
This is a pattern I want to try out.
Not sure if it's ugly yet.
2018-03-10 15:32:31 -08:00
633dbfa3cf Fix typo, searchhelpers.py is a file. 2018-03-10 14:42:33 -08:00
9355cd09ef Add digest_directory parameter new_photo_ratelimit.
Initially inspired by the nuisance of having multiple photos
with the same timestamp because the system is too fast, this
feature could also be used to lessen disk load for running in
the background.
2018-03-10 14:18:28 -08:00
c90585665a Add md and rst as text/plain mimetypes. 2018-03-10 13:27:21 -08:00
5f6d21fdee Give Photos a searchhidden property.
By default, photos with searchhidden do not appear in the search
results. This allows a small number of representative images from
a large album to appear in the results, while the rest can be
found on the album's page.

The same effect could be achieved with a tag and forbid search,
but tag searching has much higher cost and it would be more difficult
to implement as a default behavior without requiring lots of special
checks whenever listing tags etc.
2018-03-09 17:10:27 -08:00
d88db08693 Allow PDB.rollback to take a savepoint; Warn early commits.
If a @transaction method makes calls to other @transaction
methods, and one of those nested calls makes a commit, and then
the outer call raises an exception, then the outer call will
not rollback properly because its savepoint is no longer in
the savepoint stack. So let's warn the user if that happens.
Should this raise an exception instead of just warn? Not sure,
I mean the data is already committed.
2018-03-05 21:35:36 -08:00
26141f8198 Put savepoints in the on_commit_queue so we can cancel on rb. 2018-03-02 20:30:55 -08:00
8b9ef2a492 Oops, add commit parameter to Photo.copy_tags. 2018-02-25 18:56:11 -08:00
4ae5f52db3 Add some missing decorators and explain others.
Leave comments in places where @transaction or @required_feature
might be expected, but are taken care of by something else.
2018-02-25 18:55:46 -08:00
3d925c1426 Rename inner wrapped to improve traceback clarity.
Because "line 77, in wrapped_transaction" is at least slightly
better than "in wrapped".
2018-02-25 17:15:35 -08:00
c001b0fc87 Use SQL savepoints to improve @transaction.
I didn't know about these! Now we don't have to roll all the way
back in case of problems. This allows for raising exceptions as
control flow since they won't undo unrelated stuff.
2018-02-25 17:10:04 -08:00
576fac416a Move SQL heavy lifting to its own Mixin. 2018-02-25 16:30:05 -08:00
e4e542a8a6 Replace manual DELETEs with calls to sql_delete. 2018-02-25 16:17:19 -08:00
a2e13003e8 Add PhotoDB.sql_delete. 2018-02-25 16:09:36 -08:00
f34164bf85 Add search feature has_thumbnails.
Search needs a complete refactor.
But until then, let's keep adding to it!
2018-02-24 23:06:25 -08:00
d653673277 Update truthystring to return bool, not given int. 2018-02-24 23:01:44 -08:00
0228fbebfd Move the random hex generator to helpers. 2018-02-24 18:54:59 -08:00
13e8bc4a6f Add more items to to-do list; minor fixes. 2018-02-20 16:50:39 -08:00
5d6c5ef2b4 Raise exception if attempted override name is invalid.
Technically it doesn't really matter because the override is not
used for any file ops, but I don't like the idea of busted
path-looking strings being displayed anywhere on the interface.
2018-02-17 15:47:26 -08:00