Commit graph

536 commits

Author SHA1 Message Date
fa78fa3b9d
Add global ARGV_LEVEL for runtime use. 2022-04-30 07:22:18 -07:00
19f69473c8
Add AppendAction to store_types. 2022-04-30 07:21:57 -07:00
306ddf489c
Change variable name for clarity, strip out comment lines. 2022-03-28 14:18:43 -07:00
59b0916f88
Add more information when logging at DEBUG, LOUD. 2022-03-24 15:35:40 -07:00
36e4a44bfb
Filter out comment lines beginning with --. 2022-03-23 23:08:43 -07:00
956b3c6053
Handle the case in which colorama is not installed. 2022-03-23 14:38:15 -07:00
5ddc83e3ff
Call expanduser in pathclass init. 2022-03-23 14:37:04 -07:00
1026d4414c
Add a calling card to @transaction for runtime determination. 2022-03-22 22:17:32 -07:00
9c9123e2d2
Update type annotation, savepoint_id is an int. 2022-03-22 22:16:42 -07:00
a76982e581
Add type annotation sqlite3.Cursor. 2022-03-22 22:16:26 -07:00
6da730e5a6
Add worms.id_type to improve behavior of int ids.
If the project uses numerical IDs, and processes user input from a web
request URL where everything is a string, it is tedious to convert
between str and int on the application side. If done improperly, it
messes up the cache because the int id and str id will be entered
separately. So we let the database handle the id type and raise errors
if necessary.
2022-03-21 19:41:59 -07:00
8f2b0ebc6a
Add sqlhelpers.Inject. 2022-03-20 13:04:28 -07:00
629594b6ae
Add SSE functions. 2022-03-20 13:01:26 -07:00
f79fcf6432
Improve flasktools.cached_endpoint.
- Store separate states for different url and param permutations.
- Add the etag_function which allows you to decide not to run the
  function in the first place, instead of running it and seeing
  that its return value matches the previous value.
2022-03-20 13:00:46 -07:00
4b0d50c193
Add isinstance check for sqlite3.Row. 2022-03-19 20:03:50 -07:00
465f22fedb
Instead of checking get_tables, just check the COLUMN_INDEX.
This reduces the number of frivolous sql queries we make, since
the index contians all the tables anyway. The side effect is that
the function won't acknowledge tables created during the run time,
which I kind of think is a good thing since it reduces accident
surface area to the tables outlined by the programmer during init.
2022-03-19 20:03:26 -07:00
59df12a23f
Simplify worms.get_tables with select_column. 2022-03-19 19:51:25 -07:00
6867fb3011
Remove normalize_db_row, just use row_factory = Row. 2022-03-15 18:17:23 -07:00
ac647a37a8
Let comma_space_split return empty list if string is all blank. 2022-03-15 13:37:57 -07:00
5f5f41885e
Move get_tables into alphabetical order. 2022-03-15 13:37:28 -07:00
05b009a7c6
Escape backslashes when stringifying. 2022-03-15 13:29:14 -07:00
ee4914fa63
Add self.last_commit_id to help application know state of the world.
This allows caching layers to make better decisions about when to
reload data.
2022-03-14 15:37:46 -07:00
9b2ad4d027
Let delete, insert, update return the self.execute return value. 2022-03-14 15:35:51 -07:00
512f1591ef
Add select_one_value. 2022-03-14 15:34:41 -07:00
38f9d8046c
Add worms.raise_without_rollback. 2022-03-14 15:32:11 -07:00
0fd09984bc
Let object tokens repr themselves in the diagram. 2022-03-14 15:30:46 -07:00
ed952ffdda
Use getrandbits instead of passwordy strings for savepoint ids.
Better perf, why not.
2022-03-10 11:55:07 -08:00
d17a79c128
Store into cache using instance.id instead of requested id.
This helps ensure that the keys of the cache are the true and correct
data type and not just whatever the user provided. SQLite coerces
strings and ints.
2022-03-10 11:24:52 -08:00
c7f4d1dbf9
Respect the user's NO_COLOR environment variable. 2022-03-10 11:19:36 -08:00
2a6bc2a306
Include digits by default. 2022-03-10 11:18:21 -08:00
9530d0383a
Updates to downloady.
- Use new progressbars.
- Let the user pass a file handle to download bytes into.
- Use dynamic chunk sizing to get more regular progress bar updates.
2022-02-21 14:33:22 -08:00
232398eff0
Big updates to betterhelp.
Instead of handwriting the help text, which was time consuming and
prone to errors, I'm finally using the help parameter in
parser.add_argument. betterhelp will render and colorize this for
some good-looking automatic help text.

The make_helptext function is still extremely long and would be nice
to refactor, but I've been sitting on this commit for a few weeks now
and I want to get my git repositories back in sync.
2022-02-21 14:33:22 -08:00
c1cba4f287
Add replace_color. 2022-02-12 20:09:14 -08:00
797b4efd89
Let pipeable.output be a public function. 2022-02-12 20:09:14 -08:00
e34f3e6c32
Add progressbars.py.
I've been tweaking this file for weeks and I just want to publish
it now even though the code is still kind of ugly. I think the
interface is done.
2022-02-12 20:09:13 -08:00
ecd663b7e9
Some improvements to ratemeter.
threading.lock to ensure consistency
time.monotonic instead of time.time
Docstring wrapping
Remove math.ceil and round, not sure why I did that
2022-02-12 20:09:13 -08:00
f550c03c15
Replace constants IN_PIPE, OUT_PIPE with functions.
Since it's possible to monkeypatch over sys.stdout etc., it works
better to have functions instead of constants from the beginning
of runtime.
2022-02-12 20:09:12 -08:00
72efaec70c
Add niceprints.ASCII_BOX boxchars. 2022-01-19 20:57:37 -08:00
053a25cc58
Add DotDict.__iter__, yields key value pairs. 2022-01-19 20:57:04 -08:00
83a3bee53d
Update passwordy docstring formatting. 2022-01-19 20:56:40 -08:00
3a5235c636
Add --prefix, --suffix. 2022-01-19 20:29:58 -08:00
8d258241e7
Let pathclass.glob('.') return cwd.
This brings it more in line with regular glob for which glob('.') = ['.'].
2022-01-19 20:23:05 -08:00
9a0be7eda9
space message 2022-01-19 19:52:46 -08:00
bcab9b34d6
Add Path.assert_disk_space. 2022-01-19 19:23:11 -08:00
77769aa2ef
Alphabetize exceptions. 2022-01-19 18:38:05 -08:00
7fddee51df
Use sentinels for mode instead of strings. 2022-01-19 18:37:49 -08:00
0be376551f
Typecheck the numeric parameters to Ratelimiter. 2022-01-19 18:31:49 -08:00
206e29d4bd
Fix hms command line use. 2022-01-11 19:39:55 -08:00
8b4202fe87
Always use urandom in all functions. Remove --urandom option. 2022-01-09 17:05:03 -08:00
d34103917f
Re-add the normalize_sep function, but more conservative.
Formerly I was normalizing backslash to forwardslash on unix, which is
bad. Then I removed the function because I thought normpath was the
right tool for the job. Now I'm finding a few situations I just want to
norm the sep again.
2022-01-09 16:59:09 -08:00
4b8ae01786
Add starting_balance parameter, and by default make it nonzero. 2022-01-09 16:55:53 -08:00
e5114ecf2c
Alphabetize docstring. 2022-01-09 16:55:15 -08:00
c914d4236e
Make only_shrink a keyword-only argument. 2022-01-09 16:54:15 -08:00
c1886ba536
Add OVERWRITE_ sentinels for different overwrite techniques. 2022-01-04 13:07:16 -08:00
31156aa67c
Sort scandir entries.
In spinal.walk sorting is optional because there I prioritize
configurability and speed, but for pathclass.walk I think it
makes sense to prioritize comfort and just-works results.
2022-01-04 12:47:17 -08:00
d3d5bd1fa2
Add base mutable class. 2021-12-27 22:33:18 -08:00
10a312b996
Add return value written_files. 2021-12-27 17:08:54 -08:00
373d8dd36d
Improve operatornotify betterhelp epilogue. 2021-12-23 17:07:36 -08:00
90a7c4b76a
Add "or" to the title_capitalize lowercase list. 2021-12-23 17:07:22 -08:00
597705fc57
Add hms_letters functions to make 00h00m00s outputs. 2021-12-21 17:09:40 -08:00
9381c2c9c7
Also log commits without user-given messages. 2021-12-21 17:02:26 -08:00
28b4c8626b
Fix ratelimiter reject mode always subtracting balance. 2021-12-21 17:02:08 -08:00
0a0bbae88c
Use unicode_width instead of len, improve in_box. 2021-12-20 16:45:47 -08:00
ca4b78c654
Add stringtools.unicode_width. 2021-12-20 16:44:10 -08:00
a73263f27d
Log absolute_path instead of the Path object. 2021-12-20 16:43:14 -08:00
359c255817
Make pathclass globs faster by checking scandir for stat first. 2021-12-16 21:14:35 -08:00
c2dae46c99
Update interactive.py. 2021-12-14 20:37:40 -08:00
90f14e04f8
Improve a few uses of pathclass. 2021-12-07 12:41:58 -08:00
f2190b9bfb
Add Path.walk_directories, walk_files. 2021-12-07 12:41:35 -08:00
23fe4d1d1e
Call path.assert_is_directory before walking. 2021-12-02 19:32:39 -08:00
19afbc2105
Enforce with_child basename is str. 2021-12-02 19:32:25 -08:00
e2fd04ccf1
Remove .absolute_path thanks to fspath. 2021-12-02 19:32:08 -08:00
b288cca519
Rewrite a lot of pathclass, spinal.walk using tuple-based Path.
I was inspired by the idea of "making impossible states impossible"
and using a data model that accurately represents what we intend for
it to represent. Instead of storing the path as a string where "it's
a string but actually you're supposed to know that the parts between
os.seps are different parts and the first one is special and...", we
can use a data model that directly says that. Storing the path as a
tuple of (Drive, Part, Part) helps me focus on the semantics of the
Path as a collection of parts joined by the os.sep.

Furthermore, storing the path as a string made some operations slow.
Every time we call one of the os.path functions with a string, it
has to do a lot of normalization and edge-case handling even when we
know it wouldn't be needed. By storing the path as a tuple, we can
instantly get the drive name, parent dir name, and basename without
asking os.path to split it for us every single time. It also makes
relative path / common ancestor checks a lot easier to understand.
Fewer operations need to go into the slow functions.
2021-11-30 21:16:47 -08:00
7a48a7b69c
Let wait_for_internet take a backoff class to control sleeps. 2021-11-24 15:14:18 -08:00
bef06dac75
Add Pool.done, indicates all threads have completely finished. 2021-11-20 19:50:18 -08:00
9b8dd35658
Let Pool._threads be a set instead of list. 2021-11-20 19:48:12 -08:00
305c6bfdf4
Add lock around running_count increment, decrement.
I thought these assignments were atomic for int but they're not.
2021-11-20 19:48:12 -08:00
cca488bc7a
Move code to own function Pool.close. 2021-11-20 19:48:12 -08:00
87392b31a3
Add some type annotations. 2021-11-20 19:48:11 -08:00
dcd4d20b07
Rename PooledThread.start -> mainloop. 2021-11-20 19:20:39 -08:00
f1aea14646
Move the sleep out of the locked portion. 2021-11-20 18:53:56 -08:00
8f8a39eaca
Add Ratelimiter.__repr__. 2021-11-20 18:52:00 -08:00
6d8a406ceb
Use time.monotonic. 2021-11-20 18:51:46 -08:00
edeff079f7
Use vlogging. 2021-11-20 18:49:31 -08:00
8317e7c7b7
Add some docstrings. 2021-11-20 18:48:58 -08:00
6d9300aa9d
Remove task_done since we never call queue.join. 2021-11-20 18:48:40 -08:00
039ac45e17
Let Job.join accept a timeout. 2021-11-20 18:48:05 -08:00
f7250f8157
Let downloady take ratemeters to assist caller's reporting. 2021-11-20 18:46:20 -08:00
2fbe24b272
Add argument --operatornotify-subject to override subject line. 2021-11-18 19:53:54 -08:00
1c1bb49cb8
Add ----- between epilogues. 2021-11-18 17:06:00 -08:00
e995bd502d
Use os.path.normpath instead of normalize_sep. 2021-11-18 17:02:32 -08:00
f1b88b49fe
Raise typeerror if not Path or str, and dedent by early return. 2021-11-14 22:18:03 -08:00
3049a62bc0
Use httperrors. 2021-11-13 01:21:32 -08:00
ef546886c9
Use sock.settimeout instead of module setdefaulttimeout. 2021-11-13 01:20:42 -08:00
1bbb8bedee
Use voussoirkit.httperrors. 2021-11-13 01:20:26 -08:00
e8a0b0a050
Let get_external_ip take a timeout. 2021-11-13 01:20:13 -08:00
53756662ad
Disambiguate kwargs. 2021-11-11 18:17:42 -08:00
8838d2ffee
Fix capitalization in epilogue. 2021-11-10 23:41:50 -08:00
1c96931f85
Add argument thousands_separator. 2021-11-10 23:02:38 -08:00
729d3e5cf3
Improve bytestring docstring. 2021-11-10 23:02:22 -08:00