a122f72af7
Fully migrated to jsonb, adjusted uniquefifo to keep cache fresh
...
* Added fifo mechanism to actors table
* Increased fifo size to 10
* Still getting some database insert duplicate errors, but only for
very active instances when they are newly identified
2021-02-02 23:34:43 +00:00
f262de1dc3
migrating to storing data as jsonb object in database
...
captures all data, avoids cache-misses
2021-02-01 20:31:40 +00:00
3662535b0d
renaming posts to activities and accounts to actors
...
removed old posthash mechanism
2021-02-01 12:52:42 +00:00
7e71d0cc7a
Installed recent requests structure to prevent repeat requests
...
Default size is 5
Only targets posts, not users
2021-02-01 00:28:20 +00:00
1f3cbe8bde
Isolated BuildClient from GetRunner
...
Now there is a single place where new clients are built
2021-01-30 05:54:03 +00:00
96e47f0373
Removing GetHTTPSession for GetRunner
2021-01-29 17:44:16 -05:00
4706bce7e3
hardcoded user-agent as tusky + added a DoTries()
2021-01-25 21:06:47 -05:00
1bebd9064c
removing urls from normalization
2021-01-25 20:28:13 -05:00
9d2601e6c9
Replaces <br> tags with spaces
2021-01-14 20:43:20 +00:00
1adaba8322
retrying connections + logging prefix
...
this is because some hosting providers throttle rapid new connections
regex additions
2021-01-14 19:51:42 +00:00
81b06d64e8
hack around ignoring 'to' URIs that end in /followers
...
may remove the entire associated for-loop...
2021-01-04 18:40:11 +00:00
d02ae808a1
reducing potential keep-alive connections to 1 per hostname (not per IP)
2021-01-02 07:00:21 +00:00
85e0735fa3
go fmt'ed
2020-12-29 20:20:02 +00:00
20fb4ed76a
mostly web, fixed a regression
2020-12-29 17:30:26 +00:00
5473052519
changed comments, made streaming in a loop
2020-12-29 16:38:14 +00:00
d50dc1ec0d
Cleaning up some errors
2020-12-29 15:47:49 +00:00
63bc88324b
changed to pool from connection, few error fixes
2020-12-29 15:03:52 +00:00
eafb3b9318
added normalization
2020-12-25 05:45:08 +00:00
9db3c05ed6
few bugs
2020-12-23 08:20:03 +00:00
28fa8ab5ec
deleting comments
2020-12-22 20:30:21 +00:00
39b53ed45c
added stream and poll
...
fixed tables.sql
2020-12-22 20:20:12 +00:00
6b8d6a3bef
reusing connections
2020-12-22 19:46:34 +00:00
e9178b1d09
verifying is being done
...
middle layer commit
database middle layer might be redundant
screwed up tables.sql
isn't re-useing connections
2020-12-22 17:35:05 +00:00
