4c65ccaf68
Big! Liberate Groupables from strict heirarchy. Multiple parents.
...
I found that the strict heirarchy was not satisfying the situation
where one tag is the intersection of two others, but we can only
pick one as the parent
For example, does red_jacket belong under clothes.red_clothes or
clothes.jackets? A search for "red_clothes AND jackets" might
give us someone wearing red pants and a black jacket, so this
definitely needs to be a separate tag, but picking only one
parent for it is not sufficient. Now, a search for red_clothes
and a search for jackets will both find our red_jacket photo.
The change also applies to Albums because why not, and I'm sure
a similar case can be made.
Unfortunately this means tags no longer have one true qualname.
The concept of qualnames has not been completely phased out but
it's in progress.
This commit is very big because I was not sure for a long time
whether to go through with it, and so much stuff had to change
that I don't want to go back and figure out what could be grouped
together.
2018-07-20 13:09:06 -07:00
4987db4bd0
Switch to f-string formatting in many places.
2018-07-18 18:36:36 -07:00
4a39873fed
Increase default tag cache to 10,000.
2018-07-18 18:27:18 -07:00
38bac41eb9
Add helpers.split_easybake_string for prelim parsing.
...
Helps to remove some distractions from the main easybake method.
2018-07-15 14:23:43 -07:00
f06b0915ab
Add new_photo_kwargs to the create_or_get call.
...
Due to the closure, it was already working just fine, but I
want the explicitness of having it there.
2018-07-15 13:52:08 -07:00
57a9e0d4dc
Move PDB.close out of the sqlmixin, into main class.
...
Doesn't really deserve to be with the sql stuff.
2018-07-15 13:27:04 -07:00
c37b54b456
Add dividers and rearrange some of the table column parsing.
2018-07-13 19:22:05 -07:00
0852630cd1
Alphabetize _extract_table_name.
2018-06-30 12:55:30 -07:00
c930228d3f
Rearrange order of Album.delete ops to not void FK contraint.
2018-06-30 12:51:09 -07:00
c736c33165
Add caching of tag synonyms.
2018-05-06 21:09:15 -07:00
c6b71f1a0d
Revert caching of qualified names. Causing too many problems.
...
The world is just not ready for it. Was having issues about the
parents being deleted / renamed and needing to propogate those
changes in a not-ugly way.
Will reassess in the future.
2018-05-06 21:09:09 -07:00
0c8a57b6aa
Make the assert methods public.
2018-05-06 21:09:09 -07:00
40c255b0d0
Pass object instantiations through a cacher.
...
I want the system to only have one instance of a particular
object at any time, so that we can effectively cache things
in them.
2018-05-06 21:09:08 -07:00
11fda94968
Make the SQL_COLUMNS code a bit easier to read.
2018-05-06 21:09:02 -07:00
db28b6819c
Replace individual cache instantiations with all at once.
2018-05-06 21:08:56 -07:00
96d79f2b25
Oops, let separate loading of pragmas for not-first-time load.
...
Since DB_INIT was only run during first time setup, all future
loads were not running the pragmas, not even the FK restriction.
2018-05-03 19:02:53 -07:00
64449569ea
Rearrange statements during tag delete to not voilate FK.
2018-05-03 18:59:50 -07:00
23cc769357
Take advantage of parent's cached_qualname when making mine.
2018-05-03 18:45:57 -07:00
943deb9bf7
Move child-lifting code to its own helper method.
2018-05-03 18:32:44 -07:00
ae46b097e4
Let get_tag_by_name return inputted Tag right away.
...
For caller's convenience most functions assume you are passing
name as a string, but if you already have the Tag we'll just use it.
2018-05-03 18:20:48 -07:00
a0e3e037c1
Bail out of join_group early if already immediate parent.
...
I'd rather make the extra select than a delete and reinsert.
2018-05-03 18:18:13 -07:00
22269af2ca
Always use False for leave commit before joining new group.
2018-05-03 18:10:12 -07:00
a8fc1206d9
Oops, move get_root_tags into alphabetical position.
2018-05-02 17:41:46 -07:00
58d450a856
Fix handling of the event that a file loses its thumbnail.
...
In which case return_filepath is None and the relative_to breaks.
2018-05-01 20:41:38 -07:00
1626a6fa11
Optimize PDB.get_root_albums and get_root_tags.
...
Achieves in a single query what used to require a ton of
individual get_parent calls.
2018-05-01 20:32:45 -07:00
48396d6ab6
Add Album.remove_photos for batch removals.
2018-05-01 18:05:07 -07:00
698981dd29
Move _uncache_sums call out of _add_photo, into callers.
2018-05-01 17:48:56 -07:00
fa97512586
Use batches of 999 for get_things_by_id.
...
SQLite has a limit on how many variables you can use per query.
2018-05-01 17:48:02 -07:00
97e1f881d0
Move video thumbnailing code to helpers.
2018-04-28 20:36:33 -07:00
5bc2bbdacb
Move image thumbnailing code to helpers.
2018-04-28 20:33:05 -07:00
41823394eb
Use PDB.get_albums_by_id for Photo.get_albums.
2018-04-28 19:17:11 -07:00
62496d183e
Use PDB.get_tags_by_id for Photo.get_tags.
2018-04-28 19:14:58 -07:00
b19682d1a5
Replace Photo.sorted_tags with jinja filter.
2018-04-28 19:05:51 -07:00
df7fbeae0a
Minor clarity improvements to required_feature logic.
2018-04-27 20:04:18 -07:00
e2cf9344c1
Copy the inputted new_photo_kwargs so as not to damage it.
...
Also I learned that pop takes a fallback.
2018-04-20 15:31:41 -07:00
fec5eaf21e
Add Tag.normalize_name to be called by PDB.normalize_tagname.
2018-04-20 15:28:27 -07:00
4434517c54
Rename config user.min_length -> min_username_length.
2018-04-15 14:36:07 -07:00
21bd211889
Give Users a display_name.
2018-04-15 14:23:24 -07:00
8562b355ce
Fix typo in debug commit message.
2018-04-15 13:49:10 -07:00
31b808bd64
Fix get_things_by_id not putting the results in cache.
2018-04-15 02:53:14 -07:00
7d8b8daeba
Rewrite Album.sum_bytes to use SQL sum in a single query.
2018-04-15 02:52:41 -07:00
f77ab51b14
Add Album.has_any_photo and has_any_subalbum_photo.
2018-04-15 02:41:24 -07:00
0ee07023a1
Fix ids_needed being modified while iterating.
2018-04-15 02:35:46 -07:00
518a45ccd8
Add method get_things_by_id for mass lookups.
2018-04-15 02:14:06 -07:00
68d6e4faf4
Add explicit method get_album_by_id.
2018-04-15 02:10:42 -07:00
0094e7c006
Fix bug where add_tag supersedence caused early commit.
2018-04-15 01:14:20 -07:00
bfc4f313a9
Switch some formats to f-strings.
2018-04-15 01:13:02 -07:00
028a8cb2ef
All of the INTERSECTs can just be WHERE clauses.
2018-04-04 22:08:08 -07:00
c0df14db62
Fix typo binding -> bindings.
2018-04-04 21:40:39 -07:00
abe82dde9e
Let get_tag_by_name use cache after finding the ID.
2018-03-31 14:01:03 -07:00
d9f5765bcc
Use more self.sql_ instead of own cursors.
2018-03-29 19:24:48 -07:00
144e97d365
Use self.photodb.sql_ methods instead of running own cursors.
...
All in the name of centralization.
Also improved SQLness of Tag.convert_to_synonym.
2018-03-29 19:24:48 -07:00
2edb9a1d57
Centralize cursors to PDB.sql_execute.
...
This gives me a nice common place to do logging if I want.
2018-03-29 19:24:39 -07:00
cca8837863
Move helpers.select_generator to PDB.sql_select.
2018-03-25 16:32:17 -07:00
da47b5bb3c
Fix several small, embarassing typos.
2018-03-23 01:18:43 -07:00
77dfeb4299
Add more _assert methods to hide clutter.
2018-03-23 00:35:56 -07:00
229530580f
Add Tag.normalize_description.
2018-03-23 00:35:56 -07:00
b4d3de3651
Add Bookmark.normalize_title and _url.
2018-03-23 00:35:56 -07:00
6ee86431e5
Add Album.normalize_title and _description.
2018-03-23 00:35:55 -07:00
dddd8a3aa1
Add ObjectBase.normalize_author_id.
2018-03-23 00:35:47 -07:00
1f7749b80a
PDB.new_album call album.add_photos instead of add_photo.
2018-03-22 23:11:25 -07:00
0e3ae11610
Improve normalizers, use less None; Add author search box.
...
It was getting difficult to remember which of the normalizers
use None and which don't. So let's try to be a little more consistent
and just use empty sets, etc, so the caller can rely on receiving a
set instead of having to check for None.
Also renamed search parameter authors->author to be more in line
with the singular form of extension.
2018-03-22 21:09:21 -07:00
088a79ffff
Add method PDB.get_users.
2018-03-22 21:06:05 -07:00
5c89e13129
Update DatabaseOutOfDate message to use new upgrader path.
2018-03-22 17:34:43 -07:00
5be174d1b3
Completely rewrite search to use more SQL and less application.
...
In order to achieve tag_musts, we break each of the musts down
into separate EXISTS queries for each of the matchable children.
Then we INTERSECT those, and finally do other filtering and
ordering as usual.
2018-03-21 19:20:43 -07:00
1e42f1ea4e
DB V12: Add (photoid, tagid) index to photo_tag_rel.
2018-03-21 19:00:13 -07:00
acc9b67a20
Add parameter minimal to jsonify.tag.
2018-03-21 18:48:38 -07:00
6c77ddd1c3
searchhelpers.normalize_author use un only, not IDs.
...
I don't like the idea of a user setting their username to be
someone else's ID.
2018-03-21 13:21:25 -07:00
afc5f38a8d
Combine normalize_limit, _offset to normalize_positive_integer.
2018-03-21 11:25:21 -07:00
1d76b4511b
Let PDB.get_tag convert normalization errors to NoSuch.
...
Instead of having to account for them on the other end,
since they don't care about that stuff anyway.
2018-03-21 11:03:58 -07:00
9a1caee412
Add helper function hash_file.
2018-03-18 20:56:29 -07:00
2d73e59abf
Let helpers.read_filebytes operate on pathclass Paths.
2018-03-18 20:56:08 -07:00
e186653e36
Use shutil.which to discover ffmpeg.
...
How did I miss such an easy solution for so long??
So glad I did not go the env var route.
2018-03-18 17:03:11 -07:00
84e0266f30
Add author to object jsonifiers.
2018-03-18 15:59:36 -07:00
60049c777f
Add method get_author to all objects.
2018-03-18 15:58:51 -07:00
dffde094e8
Give Albums and Tags an author_id column.
...
Keeping V11 because I hadn't published previous commits yet.
2018-03-18 15:28:26 -07:00
d6d28764c4
Oops, assign self.config before dumping it.
2018-03-18 00:23:05 -07:00
6add02c8a2
Move column extractor to own function.
2018-03-18 00:21:03 -07:00
d383000441
Add Foreign Keys, Primary Keys, and NOT NULL constraints.
2018-03-18 00:09:08 -07:00
33d506ddaf
Rearrange tables in preparation for foreign keys.
...
In order to create a foreign key the table has to already exist.
So move the object types to the top, with Users first since other
things have authors.
2018-03-17 17:54:15 -07:00
c4d0de7f97
Move digest_directory and easybake to Util mixin.
2018-03-17 17:19:07 -07:00
87886a0a13
Unprivatize PDB._load_config and _save_config.
2018-03-17 17:07:30 -07:00
941d50eae0
Move config saving to own method.
2018-03-17 17:05:34 -07:00
269cfb6762
Improve some variable names for clarity.
2018-03-17 16:48:09 -07:00
ffe0be21f5
Fix Photo thumbnail paths to be relative to the PDB's thumb dir.
...
Previously they were relative to the directory from which the client
was launched, meaning it usually started with ./_etiquette.
Now it should be more portable and less tight.
2018-03-13 02:50:54 -07:00
44d1fb2fff
Add PhotoDB init option skip_version_check.
2018-03-13 02:25:04 -07:00
d85d911b22
Add PhotoDB init option create. Disable to require existing.
2018-03-13 02:24:11 -07:00
101f6d9c47
Add exception BadDataDirectory.
2018-03-13 02:21:35 -07:00
b89da5dec2
Add hasattr checks to prevent faulty __del__.
2018-03-13 02:20:17 -07:00
1cd78a678b
Minor linter appeasements.
2018-03-11 01:54:59 -08:00
c99e783d04
Rename comprehension variable to not look like a conflict.
...
The existing code worked fine, but it was odd to see that `tag`
was both the loop variable as well as the key into the children.
So let's change it.
2018-03-11 01:35:35 -08:00
a86c27ceb4
Simplify some unnecessarily detailed log formats.
2018-03-11 01:34:32 -08:00
94e518068f
Add Album.add_photos for batch adding with less savepoint waste.
...
Because each call to add_photo opens a savepoint, it is wasteful
to use it for a large number of photos that belong together.
2018-03-10 22:50:41 -08:00
f5a126d3bc
Oops, fix missing return statement.
2018-03-10 15:42:23 -08:00
42cef093a5
digest_directory: Use _normalize functions for argument prep.
...
This is a pattern I want to try out.
Not sure if it's ugly yet.
2018-03-10 15:32:31 -08:00
633dbfa3cf
Fix typo, searchhelpers.py is a file.
2018-03-10 14:42:33 -08:00
9355cd09ef
Add digest_directory parameter new_photo_ratelimit.
...
Initially inspired by the nuisance of having multiple photos
with the same timestamp because the system is too fast, this
feature could also be used to lessen disk load for running in
the background.
2018-03-10 14:18:28 -08:00
c90585665a
Add md and rst as text/plain mimetypes.
2018-03-10 13:27:21 -08:00
5f6d21fdee
Give Photos a searchhidden property.
...
By default, photos with searchhidden do not appear in the search
results. This allows a small number of representative images from
a large album to appear in the results, while the rest can be
found on the album's page.
The same effect could be achieved with a tag and forbid search,
but tag searching has much higher cost and it would be more difficult
to implement as a default behavior without requiring lots of special
checks whenever listing tags etc.
2018-03-09 17:10:27 -08:00
d88db08693
Allow PDB.rollback to take a savepoint; Warn early commits.
...
If a @transaction method makes calls to other @transaction
methods, and one of those nested calls makes a commit, and then
the outer call raises an exception, then the outer call will
not rollback properly because its savepoint is no longer in
the savepoint stack. So let's warn the user if that happens.
Should this raise an exception instead of just warn? Not sure,
I mean the data is already committed.
2018-03-05 21:35:36 -08:00
