5a16579fe3
Add a 'type' string to all etiquette jsonify outputs.
2020-04-02 22:43:20 -07:00
dd3d40de53
Experimental: Allow search results to include albums.
...
I've been thinking about this for a while but couldn't think of
the perfect way to implement it. I still haven't, so instead I'm
just starting with something and we'll see how to improve later.
At any rate, I can update the rest of the system to expect Albums
coming out of search so that if I ever have a better algorithm
everything else will already be ready for it.
For this first experiment, just any photos that are part of an album
will send that album out as a result. It doesn't even respect the
limit parameter, it's really just to see how it feels to use.
2020-04-02 22:36:40 -07:00
e94e35c98c
Add option to natural_sort filenames before digesting.
2020-04-02 21:57:54 -07:00
374763f990
Reduce log level of savepoints to 5.
...
So far I am satisfied with how they are working, and they clutter
the output so much I can't read anything else.
For reference, logging.DEBUG is 10.
2020-04-02 21:56:41 -07:00
0fdf08adf5
Fix bug where searching musts=[A, A.B] ignored B.
...
The problem was that during the expansion of A, B would get added to
the `total` set. And then because B was already in total, it would
bail early instead of expanding, but that meant it got left out of
the final query. But this only happened sometimes since random set
ordering would sometimes expand B before A and everything was ok.
I think the best solution is to do this first pass that eliminates
ancestors. Furthermore, I realized that the entirety of expand_nested
hinged on this flawed logic and was otherwise completely redundant
for a simple list comprehension.
2020-03-19 19:04:09 -07:00
13c3a1c0e9
Update docstrings for walk_children, walk_parents.
2020-03-19 19:03:47 -07:00
0a8a754cbc
Replace set([]) with {}.
2020-03-19 17:40:52 -07:00
e35efb5c61
Make this string an r-string so it gets better syntax highlighting.
2020-03-11 18:11:54 -07:00
6529aedde6
Add docstring to PDB.__init__.
2020-03-06 09:43:16 -08:00
3f2cde02e1
Use send2trash to support recycle_instead_of_delete.
2020-02-27 14:18:46 -08:00
961ece0081
digest_directory excludes extend defaults instead of overriding them.
...
With this change, there is currently no way to specify that I actually
want to include thumbs.db etc, but I think the likelikhood of wanting
to extend the defaults greatly exceeds wanting to overrid them.
2020-02-27 14:17:30 -08:00
9d620b4b97
Remove unused imports & other small things.
2020-02-20 22:44:56 -08:00
854fa4db51
Rename PDB.register_user -> new_user.
...
I was treating User a little bit special here, but I prefer to
have the more consistent terminology.
2020-02-20 00:18:40 -08:00
1fc2ed09f5
Remove commit argument from ALL methods. @transaction does it.
2020-02-19 23:56:09 -08:00
0a984117c5
Add rollback queue, like commit queue.
2020-02-19 23:46:30 -08:00
8c854f3b6a
Improve this comment about Photo rename logic.
2020-02-19 23:22:08 -08:00
f99d3d03d9
Photo.rename new_path.assert_not_exists.
2020-02-19 23:12:29 -08:00
00a443eec3
Always use the commit queue, don't bypass with commit=True.
2020-02-19 23:12:11 -08:00
7f3602d673
Remove @transaction from mixin methods, apply it to object methods.
2020-02-19 23:10:49 -08:00
953a85c998
Delete method Tag.leave_group, leftover from 1-parent days.
2020-02-19 23:05:55 -08:00
8c797024e0
Remove calls to pdb.commit(). Let @transaction do it.
2020-02-19 22:58:21 -08:00
203fb9e00f
@transaction intercept commit argument; commit / release here.
2020-02-19 22:36:00 -08:00
04f3f6f297
Remove commit=False from all method calls, it's default now.
2020-02-19 22:20:21 -08:00
3a9d7fc2dc
Set default commit=False for ALL methods.
...
I'm working on tightening up some of the transaction code. In the
past it was default commit=True because I would launch the repl, do
something, and quit, so it was nice to have auto commit. But really
it makes more sense to have it default False and be explicit when
to commit.
2020-02-19 20:08:24 -08:00
108ba8bd95
Add PDB.release_savepoint.
2020-02-19 17:00:08 -08:00
bcd13b9ae4
Make the commit queue task popping a little more clear.
2020-02-19 16:26:23 -08:00
e7c90faf79
PhotoDB.rollback without savepoint should roll all, like real sql.
...
In sqlite3, rollback without a savepoint undoes the entire pending
transaction stack, whereas I was just popping the last save.
This change makes it match real sqlite. Also the only current call
for rollback is in the @transaction decorator which is already
explicit.
2020-02-19 16:26:02 -08:00
9e0051ca9c
Photo.move_file assert new path doesn't exist yet.
2020-02-19 16:23:31 -08:00
1375d9fb26
Fix new_photo, data[extension] to use new pathclass.Extension.
2020-02-19 15:11:52 -08:00
5da8d793d6
Return this None instead of assigning then returning.
2020-02-19 14:06:53 -08:00
31517a41d2
Add missing period on exception message.
2020-02-19 13:11:57 -08:00
92f3569971
Use of extract_table_column_map has moved to sqlhelpers.
2020-02-07 15:15:49 -08:00
b88d577ddd
Use voussoirkit.imagetools for fit_into_bounds function.
2020-02-04 19:35:46 -08:00
25ed427272
Wrap DB_INIT in a transaction.
2020-02-04 18:43:30 -08:00
d6d7521bce
Assert table exists for any sql op involving argument tables.
...
At the moment, all of these functions are safe because they're
called with hardcoded tables determined by other code, not user input.
But while I was working in this area I felt it would be good to add
a safety check just in case.
2020-02-04 18:15:14 -08:00
64f9eb5f2b
Replace some % formatting with fstrings.
2020-02-04 18:12:41 -08:00
eff0efdf40
Indicate required feature in exception message.
2020-01-29 23:52:25 -08:00
2ffe3940e6
Update OutOfDate message to show actual argument.
2020-01-28 10:52:19 -08:00
93eb693bee
Separate version_pragma from other pragmas!
...
This was a dumb bug. Because the version was included in the pragmas
that get set on every load, the database was receiving the new
user_version simply by setting skip_version_check=False and letting
the regular pragmas load, so all future checks passed without
having to run the db upgrader.
2020-01-27 20:46:32 -08:00
222d63857e
Replace which with winwhich.
2020-01-21 23:50:11 -08:00
1b308d7921
Fix seconds_to_hms bug!
...
Embarrassing. If M was zero then you got HH:SS.
2020-01-06 20:21:41 -08:00
3bd8fe6392
Raise FileNotFound from read_filebytes.
2019-12-10 12:48:28 -08:00
9fa49bbddf
Add parameter only_shrink to fit_into_bounds.
2019-08-14 13:48:22 -07:00
947d05a4fa
Revert 488d793, reinstate PhotoDB.normalize_tagname.
...
In that commit, I missed three more calls in objects.py and
now I'm deciding that 5x copy-paste is not worth it.
2019-06-15 16:49:52 -07:00
bdff14ecf2
Identify a logic error without actually fixing it.
...
😎
2019-04-26 23:14:14 -07:00
cc6e7a8b16
Improve line consistency across the PhotoDB.new_* methods.
...
By clarifying why some lines are above / below the generate_id
call and putting similar code in similar order to each other.
2019-04-26 23:12:55 -07:00
a9865d8546
Replace PhotoDB.normalize_tagname with real call to Tag.norm_name.
...
I originally did this because I didn't want to accidentally call
Tag.normalize_name and forget to pass the valid parameters. However,
having this single method be on PhotoDB while the other norms are
part of their proper class has been an eyesore.
So since there are only a few calls to this I'm just inlining them
and trusting to not forget if I add more in the future.
2019-04-26 23:06:41 -07:00
8c74e42a74
Add Album.has_any_associated_directory.
2019-04-26 18:55:03 -07:00
ebde5c7ee8
Give each object class a table and centralize db_row normer.
2019-04-26 18:52:56 -07:00
fa97dd5d3c
Remove Tag._cached_synonyms. Not worthwhile.
2019-04-05 23:26:11 -07:00
d24f93809b
Add message parameter to commit instead of logging separately.
...
The upside is that we can get rid of some redundancy and reduce
the friction of adding more commit messages.
The downside of this is that the log statement always reports from
commit, instead of the function calling commit. But with unique
messages this shouldn't be too much trouble and should be worth it.
2019-04-01 23:29:01 -07:00
68a567d266
Perform is None check before detailed type checks.
2019-04-01 23:17:35 -07:00
3aa064cde3
Move log statement to top of method.
2019-04-01 23:17:01 -07:00
2f33023025
Replace singular guard continue with simple if.
2019-04-01 23:16:25 -07:00
ad140e485e
Two small docstring improvements.
2019-03-16 13:09:02 -07:00
d66dc53215
Require voussoirkit 0.0.28 for sqlhelpers.listify.
...
Instead of defining that within etiquette.
2019-01-01 18:08:47 -08:00
5d7bc1ff4b
Somewhat improve prettiness of hyphen range parse code.
2018-12-27 15:31:56 -08:00
c6c6158062
Save this assignment until it's needed.
2018-12-21 23:44:50 -08:00
7674610b88
Somewhat simplify a few conditions in photodb init.
2018-11-24 22:25:25 -08:00
1eb5a6f465
Rename DatabaseOutOfDate's parameter 'current' -> 'existing'.
2018-11-24 22:06:16 -08:00
707fdcc637
Add album cards and improve album pages.
...
- album card has placeholder for future thumbnail.
- replaced nested tree hierarchy lists with separate boxes.
- list/grid view also applies to the root listing.
- added a sticky right panel for all the tools. not pretty yet.
- mechanism for adding sticky panel changed. instead of applying
it to the #right, you apply it to #content_body so that its
grid layout can be updated properly.
2018-11-12 22:15:59 -08:00
d0208154e4
Add a function to clarify SQL_COLUMNS creation.
2018-11-11 19:08:31 -08:00
dbdd509cd2
Rearrange imports so voussoirkit is right after lib imports.
...
Added a styleguide.md file to refer back to.
Since voussoirkit is a library it feels better to have it below
the rest of the library and above the local project imports.
2018-11-04 19:27:20 -08:00
de48105e32
When thumbnailing, don't call relative_to on return.
...
The path given by make_thumbnail_filepath will already have
it in the right place, so this is probably an accident waiting
to happen.
2018-11-04 13:30:34 -08:00
25a27e6241
Move dict_to_params to flask-specific helpers.py.
...
The rest of etiquette has nothing to do with URLs so this
doesn't belong in the backend helper file.
2018-11-04 13:30:08 -08:00
20e41ad689
Extremely minor touchups.
2018-09-23 14:57:25 -07:00
95edc4d397
Add method Photo.move_file.
2018-09-23 14:54:56 -07:00
4d1575719b
Bump voussoirkit requirement to 0.0.24 for pathclass exceptions.
2018-09-23 14:54:11 -07:00
e9f567ffc9
Don't allow Groupable to add itself as a child.
...
omg what an oversight
2018-09-22 23:14:41 -07:00
670bc8b375
Allow setting searchhidden during new_photo.
2018-09-21 21:31:57 -07:00
0ccfa74709
Let helpers.generate_*_thumbnail raise FileNotFoundError.
2018-08-17 22:05:47 -07:00
bea9f905bd
Support downloading .zip of arbitrary photos, clipboard.
...
Now that creating zips of any photo set is easier, we can
let the user download whatever is on their clipboard.
2018-08-14 23:02:06 -07:00
bc6a0aa907
Improve & generalize zipfile code.
...
Moved some heavy lifting out of the flask album.zip endpoint
and into helpers.py.
Renamed some things for clarity.
2018-08-14 22:58:26 -07:00
53c86c30a1
Improve clarity of the recursive_dict_ helper functions & usage.
2018-08-11 18:19:55 -07:00
3b3c690ae3
Remove return value from load_config, so no illusion of purity.
...
This method already relies on object state anyway and writes back
to self.config, so the return value is unnecessary.
2018-08-11 18:19:10 -07:00
9eb9220ca9
Read and write the config file as UTF-8.
2018-08-11 17:57:19 -07:00
fb052a2d56
Fix Album.add_associated_directory not letting duplicates.
...
The docstring and the actual code were disagreeing.
2018-07-29 16:30:30 -07:00
30e3aa9c6f
Simplify and improve rollback logic.
2018-07-29 16:28:57 -07:00
30d96139c2
Remove commit param from PDB.sql_* methods nobody was using.
...
And is better left to the caller anyway.
2018-07-29 16:05:49 -07:00
9328600355
Add message parameter to PDB.savepoint for better logging.
...
Helps to have a visual on which methods are creating the saves.
2018-07-29 16:03:04 -07:00
56e7472c93
Revert f-strings to %s for logging calls.
...
In my excitement I forgot about that rule.
2018-07-29 16:01:26 -07:00
8447fb3343
Make deleting and unlinking tags require confirmation.
...
Added text inside the delete buttons and made them confirm.
2018-07-28 20:25:25 -07:00
1b90850818
Oops, add the required_feature decorators to remove_child.
2018-07-22 21:06:16 -07:00
2a45f4e17e
Add parameter albums to purge_empty_albums as starting set.
...
Like purge_deleted_files, you can provide a list of albums to
act as the starting point instead of letting it use the default
of all albums.
2018-07-22 20:21:00 -07:00
8a12a24e8e
Add tag_autocomplete.js.
...
Instead of embedding the entire tag list in the search.html template
every single time, this script loads the tags from the new,
cache-enabled endpoint /all_tags.json. Then we can use html5
datalists to create autocomplete forms on the search and photo pages.
2018-07-22 20:19:05 -07:00
bbf07f4401
Fix image thumbnail wrong variable bug.
...
How long has that been there?
2018-07-22 20:19:05 -07:00
d740e6d686
Oops, fix read_filebytes not stopping at the range_max properly.
2018-07-22 20:18:58 -07:00
4c65ccaf68
Big! Liberate Groupables from strict heirarchy. Multiple parents.
...
I found that the strict heirarchy was not satisfying the situation
where one tag is the intersection of two others, but we can only
pick one as the parent
For example, does red_jacket belong under clothes.red_clothes or
clothes.jackets? A search for "red_clothes AND jackets" might
give us someone wearing red pants and a black jacket, so this
definitely needs to be a separate tag, but picking only one
parent for it is not sufficient. Now, a search for red_clothes
and a search for jackets will both find our red_jacket photo.
The change also applies to Albums because why not, and I'm sure
a similar case can be made.
Unfortunately this means tags no longer have one true qualname.
The concept of qualnames has not been completely phased out but
it's in progress.
This commit is very big because I was not sure for a long time
whether to go through with it, and so much stuff had to change
that I don't want to go back and figure out what could be grouped
together.
2018-07-20 13:09:06 -07:00
4987db4bd0
Switch to f-string formatting in many places.
2018-07-18 18:36:36 -07:00
4a39873fed
Increase default tag cache to 10,000.
2018-07-18 18:27:18 -07:00
38bac41eb9
Add helpers.split_easybake_string for prelim parsing.
...
Helps to remove some distractions from the main easybake method.
2018-07-15 14:23:43 -07:00
f06b0915ab
Add new_photo_kwargs to the create_or_get call.
...
Due to the closure, it was already working just fine, but I
want the explicitness of having it there.
2018-07-15 13:52:08 -07:00
57a9e0d4dc
Move PDB.close out of the sqlmixin, into main class.
...
Doesn't really deserve to be with the sql stuff.
2018-07-15 13:27:04 -07:00
c37b54b456
Add dividers and rearrange some of the table column parsing.
2018-07-13 19:22:05 -07:00
0852630cd1
Alphabetize _extract_table_name.
2018-06-30 12:55:30 -07:00
c930228d3f
Rearrange order of Album.delete ops to not void FK contraint.
2018-06-30 12:51:09 -07:00
c736c33165
Add caching of tag synonyms.
2018-05-06 21:09:15 -07:00
c6b71f1a0d
Revert caching of qualified names. Causing too many problems.
...
The world is just not ready for it. Was having issues about the
parents being deleted / renamed and needing to propogate those
changes in a not-ugly way.
Will reassess in the future.
2018-05-06 21:09:09 -07:00
0c8a57b6aa
Make the assert methods public.
2018-05-06 21:09:09 -07:00
40c255b0d0
Pass object instantiations through a cacher.
...
I want the system to only have one instance of a particular
object at any time, so that we can effectively cache things
in them.
2018-05-06 21:09:08 -07:00
11fda94968
Make the SQL_COLUMNS code a bit easier to read.
2018-05-06 21:09:02 -07:00
db28b6819c
Replace individual cache instantiations with all at once.
2018-05-06 21:08:56 -07:00
96d79f2b25
Oops, let separate loading of pragmas for not-first-time load.
...
Since DB_INIT was only run during first time setup, all future
loads were not running the pragmas, not even the FK restriction.
2018-05-03 19:02:53 -07:00
64449569ea
Rearrange statements during tag delete to not voilate FK.
2018-05-03 18:59:50 -07:00
23cc769357
Take advantage of parent's cached_qualname when making mine.
2018-05-03 18:45:57 -07:00
943deb9bf7
Move child-lifting code to its own helper method.
2018-05-03 18:32:44 -07:00
ae46b097e4
Let get_tag_by_name return inputted Tag right away.
...
For caller's convenience most functions assume you are passing
name as a string, but if you already have the Tag we'll just use it.
2018-05-03 18:20:48 -07:00
a0e3e037c1
Bail out of join_group early if already immediate parent.
...
I'd rather make the extra select than a delete and reinsert.
2018-05-03 18:18:13 -07:00
22269af2ca
Always use False for leave commit before joining new group.
2018-05-03 18:10:12 -07:00
a8fc1206d9
Oops, move get_root_tags into alphabetical position.
2018-05-02 17:41:46 -07:00
58d450a856
Fix handling of the event that a file loses its thumbnail.
...
In which case return_filepath is None and the relative_to breaks.
2018-05-01 20:41:38 -07:00
1626a6fa11
Optimize PDB.get_root_albums and get_root_tags.
...
Achieves in a single query what used to require a ton of
individual get_parent calls.
2018-05-01 20:32:45 -07:00
48396d6ab6
Add Album.remove_photos for batch removals.
2018-05-01 18:05:07 -07:00
698981dd29
Move _uncache_sums call out of _add_photo, into callers.
2018-05-01 17:48:56 -07:00
fa97512586
Use batches of 999 for get_things_by_id.
...
SQLite has a limit on how many variables you can use per query.
2018-05-01 17:48:02 -07:00
97e1f881d0
Move video thumbnailing code to helpers.
2018-04-28 20:36:33 -07:00
5bc2bbdacb
Move image thumbnailing code to helpers.
2018-04-28 20:33:05 -07:00
41823394eb
Use PDB.get_albums_by_id for Photo.get_albums.
2018-04-28 19:17:11 -07:00
62496d183e
Use PDB.get_tags_by_id for Photo.get_tags.
2018-04-28 19:14:58 -07:00
b19682d1a5
Replace Photo.sorted_tags with jinja filter.
2018-04-28 19:05:51 -07:00
df7fbeae0a
Minor clarity improvements to required_feature logic.
2018-04-27 20:04:18 -07:00
e2cf9344c1
Copy the inputted new_photo_kwargs so as not to damage it.
...
Also I learned that pop takes a fallback.
2018-04-20 15:31:41 -07:00
fec5eaf21e
Add Tag.normalize_name to be called by PDB.normalize_tagname.
2018-04-20 15:28:27 -07:00
4434517c54
Rename config user.min_length -> min_username_length.
2018-04-15 14:36:07 -07:00
21bd211889
Give Users a display_name.
2018-04-15 14:23:24 -07:00
8562b355ce
Fix typo in debug commit message.
2018-04-15 13:49:10 -07:00
31b808bd64
Fix get_things_by_id not putting the results in cache.
2018-04-15 02:53:14 -07:00
7d8b8daeba
Rewrite Album.sum_bytes to use SQL sum in a single query.
2018-04-15 02:52:41 -07:00
f77ab51b14
Add Album.has_any_photo and has_any_subalbum_photo.
2018-04-15 02:41:24 -07:00
0ee07023a1
Fix ids_needed being modified while iterating.
2018-04-15 02:35:46 -07:00
518a45ccd8
Add method get_things_by_id for mass lookups.
2018-04-15 02:14:06 -07:00
68d6e4faf4
Add explicit method get_album_by_id.
2018-04-15 02:10:42 -07:00
0094e7c006
Fix bug where add_tag supersedence caused early commit.
2018-04-15 01:14:20 -07:00
bfc4f313a9
Switch some formats to f-strings.
2018-04-15 01:13:02 -07:00
028a8cb2ef
All of the INTERSECTs can just be WHERE clauses.
2018-04-04 22:08:08 -07:00
c0df14db62
Fix typo binding -> bindings.
2018-04-04 21:40:39 -07:00
abe82dde9e
Let get_tag_by_name use cache after finding the ID.
2018-03-31 14:01:03 -07:00
d9f5765bcc
Use more self.sql_ instead of own cursors.
2018-03-29 19:24:48 -07:00
144e97d365
Use self.photodb.sql_ methods instead of running own cursors.
...
All in the name of centralization.
Also improved SQLness of Tag.convert_to_synonym.
2018-03-29 19:24:48 -07:00
2edb9a1d57
Centralize cursors to PDB.sql_execute.
...
This gives me a nice common place to do logging if I want.
2018-03-29 19:24:39 -07:00
cca8837863
Move helpers.select_generator to PDB.sql_select.
2018-03-25 16:32:17 -07:00
da47b5bb3c
Fix several small, embarassing typos.
2018-03-23 01:18:43 -07:00
77dfeb4299
Add more _assert methods to hide clutter.
2018-03-23 00:35:56 -07:00
229530580f
Add Tag.normalize_description.
2018-03-23 00:35:56 -07:00
b4d3de3651
Add Bookmark.normalize_title and _url.
2018-03-23 00:35:56 -07:00
6ee86431e5
Add Album.normalize_title and _description.
2018-03-23 00:35:55 -07:00
dddd8a3aa1
Add ObjectBase.normalize_author_id.
2018-03-23 00:35:47 -07:00
1f7749b80a
PDB.new_album call album.add_photos instead of add_photo.
2018-03-22 23:11:25 -07:00
0e3ae11610
Improve normalizers, use less None; Add author search box.
...
It was getting difficult to remember which of the normalizers
use None and which don't. So let's try to be a little more consistent
and just use empty sets, etc, so the caller can rely on receiving a
set instead of having to check for None.
Also renamed search parameter authors->author to be more in line
with the singular form of extension.
2018-03-22 21:09:21 -07:00
088a79ffff
Add method PDB.get_users.
2018-03-22 21:06:05 -07:00
5c89e13129
Update DatabaseOutOfDate message to use new upgrader path.
2018-03-22 17:34:43 -07:00
5be174d1b3
Completely rewrite search to use more SQL and less application.
...
In order to achieve tag_musts, we break each of the musts down
into separate EXISTS queries for each of the matchable children.
Then we INTERSECT those, and finally do other filtering and
ordering as usual.
2018-03-21 19:20:43 -07:00
1e42f1ea4e
DB V12: Add (photoid, tagid) index to photo_tag_rel.
2018-03-21 19:00:13 -07:00
acc9b67a20
Add parameter minimal to jsonify.tag.
2018-03-21 18:48:38 -07:00
6c77ddd1c3
searchhelpers.normalize_author use un only, not IDs.
...
I don't like the idea of a user setting their username to be
someone else's ID.
2018-03-21 13:21:25 -07:00
afc5f38a8d
Combine normalize_limit, _offset to normalize_positive_integer.
2018-03-21 11:25:21 -07:00
1d76b4511b
Let PDB.get_tag convert normalization errors to NoSuch.
...
Instead of having to account for them on the other end,
since they don't care about that stuff anyway.
2018-03-21 11:03:58 -07:00
9a1caee412
Add helper function hash_file.
2018-03-18 20:56:29 -07:00
2d73e59abf
Let helpers.read_filebytes operate on pathclass Paths.
2018-03-18 20:56:08 -07:00
e186653e36
Use shutil.which to discover ffmpeg.
...
How did I miss such an easy solution for so long??
So glad I did not go the env var route.
2018-03-18 17:03:11 -07:00
84e0266f30
Add author to object jsonifiers.
2018-03-18 15:59:36 -07:00
60049c777f
Add method get_author to all objects.
2018-03-18 15:58:51 -07:00
dffde094e8
Give Albums and Tags an author_id column.
...
Keeping V11 because I hadn't published previous commits yet.
2018-03-18 15:28:26 -07:00
d6d28764c4
Oops, assign self.config before dumping it.
2018-03-18 00:23:05 -07:00
6add02c8a2
Move column extractor to own function.
2018-03-18 00:21:03 -07:00
d383000441
Add Foreign Keys, Primary Keys, and NOT NULL constraints.
2018-03-18 00:09:08 -07:00
33d506ddaf
Rearrange tables in preparation for foreign keys.
...
In order to create a foreign key the table has to already exist.
So move the object types to the top, with Users first since other
things have authors.
2018-03-17 17:54:15 -07:00
c4d0de7f97
Move digest_directory and easybake to Util mixin.
2018-03-17 17:19:07 -07:00
87886a0a13
Unprivatize PDB._load_config and _save_config.
2018-03-17 17:07:30 -07:00
941d50eae0
Move config saving to own method.
2018-03-17 17:05:34 -07:00
269cfb6762
Improve some variable names for clarity.
2018-03-17 16:48:09 -07:00
ffe0be21f5
Fix Photo thumbnail paths to be relative to the PDB's thumb dir.
...
Previously they were relative to the directory from which the client
was launched, meaning it usually started with ./_etiquette.
Now it should be more portable and less tight.
2018-03-13 02:50:54 -07:00
44d1fb2fff
Add PhotoDB init option skip_version_check.
2018-03-13 02:25:04 -07:00
d85d911b22
Add PhotoDB init option create. Disable to require existing.
2018-03-13 02:24:11 -07:00
101f6d9c47
Add exception BadDataDirectory.
2018-03-13 02:21:35 -07:00
b89da5dec2
Add hasattr checks to prevent faulty __del__.
2018-03-13 02:20:17 -07:00
1cd78a678b
Minor linter appeasements.
2018-03-11 01:54:59 -08:00
c99e783d04
Rename comprehension variable to not look like a conflict.
...
The existing code worked fine, but it was odd to see that `tag`
was both the loop variable as well as the key into the children.
So let's change it.
2018-03-11 01:35:35 -08:00
a86c27ceb4
Simplify some unnecessarily detailed log formats.
2018-03-11 01:34:32 -08:00
94e518068f
Add Album.add_photos for batch adding with less savepoint waste.
...
Because each call to add_photo opens a savepoint, it is wasteful
to use it for a large number of photos that belong together.
2018-03-10 22:50:41 -08:00
f5a126d3bc
Oops, fix missing return statement.
2018-03-10 15:42:23 -08:00
42cef093a5
digest_directory: Use _normalize functions for argument prep.
...
This is a pattern I want to try out.
Not sure if it's ugly yet.
2018-03-10 15:32:31 -08:00
633dbfa3cf
Fix typo, searchhelpers.py is a file.
2018-03-10 14:42:33 -08:00
9355cd09ef
Add digest_directory parameter new_photo_ratelimit.
...
Initially inspired by the nuisance of having multiple photos
with the same timestamp because the system is too fast, this
feature could also be used to lessen disk load for running in
the background.
2018-03-10 14:18:28 -08:00
c90585665a
Add md and rst as text/plain mimetypes.
2018-03-10 13:27:21 -08:00
5f6d21fdee
Give Photos a searchhidden property.
...
By default, photos with searchhidden do not appear in the search
results. This allows a small number of representative images from
a large album to appear in the results, while the rest can be
found on the album's page.
The same effect could be achieved with a tag and forbid search,
but tag searching has much higher cost and it would be more difficult
to implement as a default behavior without requiring lots of special
checks whenever listing tags etc.
2018-03-09 17:10:27 -08:00
d88db08693
Allow PDB.rollback to take a savepoint; Warn early commits.
...
If a @transaction method makes calls to other @transaction
methods, and one of those nested calls makes a commit, and then
the outer call raises an exception, then the outer call will
not rollback properly because its savepoint is no longer in
the savepoint stack. So let's warn the user if that happens.
Should this raise an exception instead of just warn? Not sure,
I mean the data is already committed.
2018-03-05 21:35:36 -08:00
26141f8198
Put savepoints in the on_commit_queue so we can cancel on rb.
2018-03-02 20:30:55 -08:00
8b9ef2a492
Oops, add commit parameter to Photo.copy_tags.
2018-02-25 18:56:11 -08:00
4ae5f52db3
Add some missing decorators and explain others.
...
Leave comments in places where @transaction or @required_feature
might be expected, but are taken care of by something else.
2018-02-25 18:55:46 -08:00
3d925c1426
Rename inner wrapped to improve traceback clarity.
...
Because "line 77, in wrapped_transaction" is at least slightly
better than "in wrapped".
2018-02-25 17:15:35 -08:00
c001b0fc87
Use SQL savepoints to improve @transaction.
...
I didn't know about these! Now we don't have to roll all the way
back in case of problems. This allows for raising exceptions as
control flow since they won't undo unrelated stuff.
2018-02-25 17:10:04 -08:00
576fac416a
Move SQL heavy lifting to its own Mixin.
2018-02-25 16:30:05 -08:00
e4e542a8a6
Replace manual DELETEs with calls to sql_delete.
2018-02-25 16:17:19 -08:00
a2e13003e8
Add PhotoDB.sql_delete.
2018-02-25 16:09:36 -08:00
f34164bf85
Add search feature has_thumbnails.
...
Search needs a complete refactor.
But until then, let's keep adding to it!
2018-02-24 23:06:25 -08:00
d653673277
Update truthystring to return bool, not given int.
2018-02-24 23:01:44 -08:00
0228fbebfd
Move the random hex generator to helpers.
2018-02-24 18:54:59 -08:00
13e8bc4a6f
Add more items to to-do list; minor fixes.
2018-02-20 16:50:39 -08:00
5d6c5ef2b4
Raise exception if attempted override name is invalid.
...
Technically it doesn't really matter because the override is not
used for any file ops, but I don't like the idea of busted
path-looking strings being displayed anywhere on the interface.
2018-02-17 15:47:26 -08:00
606a66cbf9
Remove Photo.filepath and real_filepath in favor of real_path.
2018-02-17 15:19:36 -08:00
42728d5d67
Add method Photo.set_override_filename.
2018-02-17 15:07:26 -08:00
f7bd49b2dd
Add call to remove_control_chars in remove_bad_pathchars.
2018-02-17 15:07:12 -08:00
0d6faa8f6e
Take advantage of new sql_update to update where_keys.
2018-02-17 14:02:11 -08:00
9939f5da1d
For sql_insert / _update, let default commit=False.
...
Because all the calls are using it that way anyway.
2018-02-17 14:01:12 -08:00
6363cb5fc0
Rename Album.associated_directories -> get_associated_directories.
2018-02-17 13:33:27 -08:00
b4a28f5034
Make Photo.bytestring a @property.
...
This is the kind of thing that should be a @property, because
it's very low processing power and doesn't hit the database.
2018-02-16 23:16:02 -08:00
bbe392ac09
Rename Photo.author -> get_author; no longer a @property.
2018-02-16 23:11:25 -08:00
205c49c21f
Rename Photo.albums -> get_containing_albums.
2018-02-16 23:08:44 -08:00
2f9b51b636
Rename Photo.tags -> get_tags.
2018-02-16 23:07:21 -08:00
ac9d7ede22
Rename Album.photos -> get_photos.
2018-02-16 23:03:54 -08:00
6574450ad1
Rename Tag.synonyms -> get_synonyms.
2018-02-16 23:01:07 -08:00
fdcf07940a
Rename Groupable.children -> get_children.
2018-02-16 23:00:43 -08:00
75effafaf5
Rename Groupable.parent -> get_parent.
...
After making the "if x.parent is None" mistake too many times,
forgetting that it was not a @property, I've decided that the
verby name is better.
2018-02-16 22:59:34 -08:00
db92951ad6
Oops, remove incorrect name= from new_tag.
2018-02-16 22:57:35 -08:00
3c4f69f647
Specify name= on calls to PDB.get_tag.
...
Sometimes it's easy to forget whether its the name or ID,
and I like the consistency of specifying.
2018-02-16 22:25:56 -08:00
16ac6f8b85
Move UN / PW validation to helper _assert methods.
2018-02-16 22:18:19 -08:00
173c07a0bf
Add some more exception hierarchy. Exists / InvalidUN/PW.
2018-02-16 22:13:54 -08:00
1c1e7f158f
Minor fixes & pylint appeasements.
2018-02-16 21:28:36 -08:00
e5a316ff9c
Remove old SQL_ constants. Use the dicts.
2018-02-16 20:40:58 -08:00
a8ead9c0c9
Replace a lot of manual insert/update with helper.
2018-02-16 20:19:18 -08:00
3e5031a1c5
Add methods sql_insert, sql_update to centralize sqlhelper work.
2018-02-16 18:49:02 -08:00
655d1fa235
Prepare to use dicts SQL_COLUMNS, _INDEX instead of separate.
2018-02-16 18:48:03 -08:00
4e8a67aa54
Avoid sql indexing by instantiating User earlier.
2018-02-16 18:40:57 -08:00
bc4e228a25
Remove a lot of the commented code from earlier search.
2018-02-04 15:11:46 -08:00
f7dcb489c5
Remove unnecessary intification of timestamps.
...
I decided I want to have higher precision especially because
batch inserts were having several photos with the same
created timestamp making orderby >= dumb.
2018-02-03 01:35:32 -08:00
dec28b321a
Rename recursive_dict_update's parameters for clarity.
2018-02-03 01:10:54 -08:00
f266e1c79c
Split PhotoDB's large init into some smaller portions.
2018-02-03 01:10:04 -08:00
f9ee56187c
Rename attr database_file to database_filepath.
...
To make it more consistent with config_filepath and show
that it's a path object.
2018-01-30 19:46:43 -08:00
c62e6cd80b
Fix synonym fetcher using outdated column name.
2018-01-13 20:02:49 -08:00
14750eafc5
Move DB_INIT to constants; auto parse column names.
2018-01-13 16:48:48 -08:00
178a7df0b3
Add some more log debugs.
2018-01-11 19:32:15 -08:00
b5902ba4f1
Integrate voussoirkit sqlhelpers.
...
https://github.com/voussoir/else/tree/master/SQLHelpers
2018-01-09 21:15:50 -08:00
6f371701e4
Fix digest second-scan bug; Break down into smaller functions.
...
There was a bug where moving an album out of its determined parent
caused future digests to fail because I wanted to use the GroupExists
for control flow, but due to the @transaction decorator it was rolling
back the changes.
Moved some of the logical pieces of the function into subfunctions
as a visual and readability aid.
2018-01-09 18:59:15 -08:00
d548e09a96
Add .nfo to the custom mimetypes as text/plain.
2018-01-09 18:46:32 -08:00
b859dbd22a
Add bookmark cache, replace get_bookmark code with call to get_thing.
2017-12-20 20:10:58 -08:00
0b5f736574
Few minor text fixes.
2017-12-16 03:45:07 -08:00
13803dc7f7
Oops, put remove_path_badchars where it belongs alphabetically.
2017-12-07 21:37:55 -08:00
f39717c216
Minor improvements to other helpers and their docstrings.
2017-12-07 21:32:37 -08:00
0d0354f4da
Rename normalize_filepath to remove_path_badchars.
...
Because the function does not do any resolving or converting
to an absolute path, I feel the name 'normalize' is inaccurate
and could be misleading when read. This is more clear.
2017-12-07 21:15:10 -08:00
6cf355d7ce
Rename comma_split to comma_space_split to better describe behavior.
2017-12-07 20:58:06 -08:00
a7b5a7d42a
Multiple small cleanups, docstrings, messages, lint improvements.
2017-11-27 15:56:16 -08:00
e6a50500b9
Fix missing file_read_chunk from default config
2017-11-26 02:37:33 -08:00
f052858d5f
Improve log creation and use custom formatter.
2017-11-26 02:37:11 -08:00
be35f3bb1f
Fix Tag.rename from raising TagExists for old instead of desired new.
2017-11-23 21:58:38 -08:00
64d8410977
No need for self.cur, just cur.
2017-11-16 18:03:51 -08:00
259c9ee1ab
Add PhotoDB init arg ephemeral. Uses :memory: sql and tempdir.
...
Primarily necessitated by unit testing. Running through the DB_INIT
is quite slow on disk, so this argument causes the sql to be done
on an in-memory database and all the other files are put into a
TemporaryDirectory.
Eventually I would like to have the other files be in-memory too
but that may be overcomplicated and underuseful.
2017-11-16 18:02:05 -08:00
74f4e74bdf
Add a few docstrings and update readme to-do list.
2017-11-16 16:46:39 -08:00
8f000543ea
Add exceptions.DatabaseOutOfDate instead of just using an error string.
...
And don't raise SystemExit. Not sure what I was thinking.
2017-11-16 16:33:40 -08:00
009c0dc678
Allow the data_directory to contain colons (fully qualified paths)
2017-11-16 16:32:05 -08:00
73d2ef5bbf
Change exception __str__ to be the type + message
...
Because the repr is ugly
2017-11-16 16:31:10 -08:00
a510c7b55c
Improve Album bytes caching, start caching photo count.
...
More careful uncaching of the summed bytes, to minimize
recalculation. Fewer cases where the album itself
is removed from the photodb's getter cache.
This also helps the download link on album pages disappear if
the child albums don't actually have any photos.
2017-11-12 15:21:53 -08:00
96856e9361
Cleanup: More minor dusting, commenting, clarity renaming.
2017-11-11 22:49:03 -08:00
bb5fa816d8
Cleanup: Rename GroupableMixin.add to add_child for clarity.
...
Especially since for albums it might sound like you're adding a photo.
2017-11-11 22:41:26 -08:00
3f69a2c240
Cleanup: Replace "objectid" with "object_id" in the Python portions.
2017-11-11 22:38:51 -08:00
a4b875b9ff
Fix config file not being updated when the default updates.
...
The len check was incorrect ever since I started nesting keys.
Now we measure all of the keys recursively.
2017-10-05 18:39:34 -07:00
a0f68fa858
Add .opus to the extended mimetypes.
2017-10-04 16:54:28 -07:00
121f8329fb
improve normalize_authors text
2017-09-29 14:32:35 -07:00
0849796986
Make build_query params all optional, more kwargy.
2017-09-23 11:22:05 -07:00
855e94ed87
Make wheres a set instead of list.
2017-09-23 11:20:38 -07:00
163e960bfa
Fix unrecognized orderby not using the warning bag.
2017-09-23 11:08:19 -07:00
557f78eb02
Integrate author search into the query instead of postprocessing.
...
Much faster this way.
2017-09-23 11:04:26 -07:00
38e72c0c3a
Make notnulls set instead of list.
2017-09-23 11:00:37 -07:00
26a034c9a0
Make Photo.author a @property; None if no author.
2017-09-23 10:58:04 -07:00
89d9c1d893
Move the new photo creation out of the exception handler.
...
I don't like the "during the handling of the above exception" because
it's misleading as to the source of the problem.
2017-08-01 18:13:54 -07:00
bd847159fa
Organize config by nesting related stuff.
...
I thought about switching to .ini files but that's even less desirable.
2017-07-29 16:23:15 -07:00
4fc203634c
Add new config "enable_login".
2017-07-20 23:25:23 -07:00
1784e584e2
Don't let Bookmark.edit with a blank url.
2017-07-13 22:58:02 -07:00
fcd507e337
Add new_photo_kwargs param to digest_directory.
2017-06-16 02:23:48 -07:00
707a9ab7c2
Fix a few bugs when photo.bytes is None.
2017-06-15 23:08:20 -07:00
6bed300496
Add more feature_required decorators and config.
2017-06-14 23:15:47 -07:00
a7cb922983
Fix missing exceptions import
2017-06-14 22:47:50 -07:00
bc5df9f1c2
Add max_len parameter to Tag.qualified_name.
2017-06-13 20:11:30 -07:00
8a0bbbcc56
Add /bookmarks/create_bookmark endpoint. No interface yet.
2017-06-12 20:15:02 -07:00
ac2b820a30
Add PDBTagMixin.get_root_tags
2017-06-11 22:57:08 -07:00
d10fe47dc4
Add parentheses to default tag chars.
2017-06-11 22:56:21 -07:00
821a5f3371
Move export functions to their own module.
2017-06-11 22:55:45 -07:00
4f03815a30
Floating point h Ⓜ️ s, and update to-do.
2017-06-11 21:09:17 -07:00
c88654346b
Fix not jsonifying the parent album.
2017-05-13 15:44:03 -07:00
4928095abb
Minor commentary and cleanup
2017-05-13 15:27:08 -07:00
55c14f3844
Don't leave bitrate undefined for non-temporal photos.
2017-05-12 17:50:31 -07:00
a4a278c173
Give Tags a description field.
2017-05-12 17:31:17 -07:00
a485be3c64
Make bitrate a real attribute instead of calculating in jinja
2017-05-11 22:34:19 -07:00
a5924b4642
I don't know how I forgot about dict(zip())
2017-05-11 22:32:55 -07:00
091dc82f1e
Catch expressionmatch.NoTokens for filename search
2017-05-06 18:37:02 -07:00
5b7c05c39d
Improve MMF search speed by pre-filtering photo IDs
...
Not perfected yet but it's an improvement.
Waste less time checking all photos, presearch for matching photos via photo_tag_rel table.
2017-05-01 21:27:54 -07:00
fbf73ac515
Since @transaction rolls back exceptions, don't use exception-based inserts.
2017-05-01 21:26:07 -07:00
a9c7ad6993
Improve separation between front & back with etiquette_flask package
...
Move flask-specific operations out of etiquette's files and into new etiquette_flask. In etiquette_site.py, etiquette calls are fully qualified.
2017-05-01 21:23:16 -07:00
83b9adbd61
Add more uncache points to help with stale data
2017-05-01 21:16:10 -07:00
8080d92f8d
add_associated_directory checks for dupe, so remove old check
2017-05-01 20:50:52 -07:00
06522ee372
Minor cleanup
2017-05-01 20:49:59 -07:00
43bc1e74fc
Add method Album.add_associated_directory
2017-05-01 20:43:45 -07:00
09f209719b
Add decorators.@required_feature to centralize FeatureDisabled exc.
2017-05-01 20:41:56 -07:00
af6785cead
Add decorators.@transaction to rollback sql upon exception
2017-05-01 20:23:58 -07:00
6ea3e67d86
Add method associated_directories.
2017-04-22 21:39:07 -07:00
528302b674
Relax Exists params; Add the .args tuple.
2017-04-22 21:20:03 -07:00
82ca1a08da
Add superinit to the objects.
2017-04-22 21:16:30 -07:00
d95eac4bb6
Minor improvements, update to-do list.
2017-04-20 18:31:48 -07:00
e3610eef2b
Standardize NotExclusive exception usage.
2017-04-20 18:29:30 -07:00
cc0c847eb5
Make sure easybake uses the normalized name in return notes.
2017-04-20 18:28:11 -07:00
