This website requires JavaScript.
c827375640
Adding fedilogue_test.go with initial tests
Farhan Khan
2021-08-07 19:45:04 +00:00
e6e3cc1196
Adding prefix /api/v1/
Farhan Khan
2021-07-01 18:37:47 +00:00
24a4ace616
Adding trending to restapi
Farhan Khan
2021-06-30 22:28:53 +00:00
395280c600
Updated restapi queries to properly sort and offset results
Farhan Khan
2021-06-30 15:00:08 +00:00
10ad5339a4
More integration commits
Farhan Khan
2021-03-17 23:15:09 +00:00
26f56b2367
Lots of little changes, working on integration
Farhan Khan
2021-03-17 23:14:24 +00:00
1fb2577dcb
Migrating string to env variable for containerization
Farhan Khan
2021-02-16 04:08:27 +00:00
5ff1519689
initial commit of rest api code
Farhan Khan
2021-02-13 02:49:36 +00:00
b47bd9ed56
moved db conn variable to db.go
Farhan Khan
2021-02-13 02:48:11 +00:00
5a481121d2
updated indexes yet again
Farhan Khan
2021-02-13 02:46:32 +00:00
d4ef1561d4
log messages moved to logConn
Farhan Khan
2021-02-13 02:45:24 +00:00
163dee0389
removed dead code
Farhan Khan
2021-02-12 02:14:20 +00:00
cefacf7462
Status update line, disabling web server
Farhan Khan
2021-02-12 00:26:31 +00:00
947645828e
adding LogLevel setting
Farhan Khan
2021-02-11 21:05:01 +00:00
d6b03c5254
cleanup, added go mod files, table indexing properly
Farhan Khan
2021-02-11 12:34:49 +00:00
778150f83c
Rearranging files Down the line I need to move headers.go and uniquefifo.go into a module
Farhan Khan
2021-02-10 23:58:20 +00:00
126aac8c81
decoupling retrieval of actor from activity in database
Farhan Khan
2021-02-10 22:30:34 +00:00
6efd723aac
Updating table index and tsvector creation
Farhan Khan
2021-02-03 01:28:27 +00:00
9dabd87f06
forgot to use commit uniquefifo.go
Farhan Khan
2021-02-03 00:07:51 +00:00
a122f72af7
Fully migrated to jsonb, adjusted uniquefifo to keep cache fresh * Added fifo mechanism to actors table * Increased fifo size to 10 * Still getting some database insert duplicate errors, but only for very active instances when they are newly identified
Farhan Khan
2021-02-02 23:34:43 +00:00
f262de1dc3
migrating to storing data as jsonb object in database captures all data, avoids cache-misses
Farhan Khan
2021-02-01 20:31:40 +00:00
3662535b0d
renaming posts to activities and accounts to actors removed old posthash mechanism
Farhan Khan
2021-02-01 12:52:42 +00:00
102ddbe41c
Adding recent uris struct to manually added instances
Farhan Khan
2021-02-01 02:05:56 +00:00
7e71d0cc7a
Installed recent requests structure to prevent repeat requests Default size is 5 Only targets posts, not users
Farhan Khan
2021-02-01 00:28:20 +00:00
88d058528f
Changed logging to stdout
Farhan Khan
2021-02-01 00:24:35 +00:00
55cb51b94e
added or adjusted missing closes
Farhan Khan
2021-01-30 14:30:49 +00:00
cd8ecce807
Added proxy support Some IPv6 traffic still sending directly to instance not sure why not
Farhan Khan
2021-01-30 07:12:37 +00:00
2f9e0f85e4
Tor is on port 9050, Proxy commented out by default
Farhan Khan
2021-01-30 07:10:49 +00:00
1f3cbe8bde
Isolated BuildClient from GetRunner Now there is a single place where new clients are built
Farhan Khan
2021-01-30 05:54:03 +00:00
96e47f0373
Removing GetHTTPSession for GetRunner
Farhan Khan
2021-01-29 17:44:16 -05:00
4706bce7e3
hardcoded user-agent as tusky + added a DoTries()
Farhan Khan
2021-01-25 21:06:47 -05:00
1bebd9064c
removing urls from normalization
Farhan Khan
2021-01-25 20:28:13 -05:00
1203d4f164
added log.go
Farhan Khan
2021-01-17 04:17:15 +00:00
9d2601e6c9
Replaces <br> tags with spaces
Farhan Khan
2021-01-14 20:43:20 +00:00
1adaba8322
retrying connections + logging prefix this is because some hosting providers throttle rapid new connections regex additions
Farhan Khan
2021-01-14 19:51:42 +00:00
e9bd9b67cf
minor regex checking
Farhan Khan
2021-01-10 21:47:08 +00:00
ec1743905a
validating hostname before starting instance changed hostname identification from account to URI
Farhan Khan
2021-01-10 05:31:51 +00:00
48acaa5b0b
updated table constraints
Farhan Khan
2021-01-06 02:43:41 +00:00
5ea270f9ec
added table relationship
Farhan Khan
2021-01-05 05:54:27 +00:00
81b06d64e8
hack around ignoring 'to' URIs that end in /followers may remove the entire associated for-loop...
Farhan Khan
2021-01-04 18:40:11 +00:00
d02ae808a1
reducing potential keep-alive connections to 1 per hostname (not per IP)
Farhan Khan
2021-01-02 07:00:21 +00:00
7973cc9600
go report card
Farhan Khan
2020-12-29 20:21:38 +00:00
85e0735fa3
go fmt'ed
Farhan Khan
2020-12-29 20:20:02 +00:00
b2974b7501
added crawling for web reciever fixed crawling settings condition
Farhan Khan
2020-12-29 20:06:18 +00:00
20fb4ed76a
mostly web, fixed a regression
Farhan Khan
2020-12-29 17:30:26 +00:00
5914fc0890
added stream retry, probably a bad approach
Farhan Khan
2020-12-29 16:41:43 +00:00
5473052519
changed comments, made streaming in a loop
Farhan Khan
2020-12-29 16:38:14 +00:00
72821c4641
Banned sinblr.com
Farhan Khan
2020-12-29 16:33:14 +00:00
d50dc1ec0d
Cleaning up some errors
Farhan Khan
2020-12-29 15:47:49 +00:00
c93dd89332
blocking twitiverse.com by default
Farhan Khan
2020-12-29 15:05:57 +00:00
63bc88324b
changed to pool from connection, few error fixes
Farhan Khan
2020-12-29 15:03:52 +00:00
eafb3b9318
added normalization
Farhan Khan
2020-12-25 05:45:08 +00:00
88c074f76b
reduced the number of new connections to 1, reduce spamming servers set keep-alive to 2 hours probably a bunch of regressions but I don't have unit tests yet
Farhan Khan
2020-12-25 05:18:42 +00:00
9db3c05ed6
few bugs
Farhan Khan
2020-12-23 08:20:03 +00:00
777120518a
removed dead code + channel ctl is broken
Farhan Khan
2020-12-22 20:36:37 +00:00
28fa8ab5ec
deleting comments
Farhan Khan
2020-12-22 20:30:21 +00:00
39b53ed45c
added stream and poll fixed tables.sql
Farhan Khan
2020-12-22 20:20:12 +00:00
6b8d6a3bef
reusing connections
Farhan Khan
2020-12-22 19:46:34 +00:00
e9178b1d09
verifying is being done middle layer commit database middle layer might be redundant screwed up tables.sql isn't re-useing connections
Farhan Khan
2020-12-22 17:35:05 +00:00
ad9fe9fad2
checking user and made 'pool' global
Farhan Khan
2020-12-18 06:06:32 +00:00
5bc9a68d53
web feature fetching user icon adding database pool to all functions to verify before fetching
Farhan Khan
2020-12-17 18:26:55 +00:00
726ac7e0bd
formatting and linting
Farhan Khan
2020-12-17 04:23:25 +00:00
8d56747f1c
Updating Makefile and fedictl sharing same header file
Farhan Khan
2020-12-17 01:43:00 +00:00
20bb0ef788
Added license to README
Farhan Khan
2020-12-17 01:39:57 +00:00
394270b1d3
minor edit to readme
Farhan Khan
2020-12-17 01:38:59 +00:00
cc020bb6f0
Merged the two directories, no need to keep them separate
Farhan Khan
2020-12-17 01:37:34 +00:00
064acdcf2d
client added to runninginstances, can be shared across goroutines adding basic post verification scaffolding
Farhan Khan
2020-12-17 01:15:17 +00:00
6c6d23a340
minor edit of README.md
Farhan Khan
2020-12-17 01:14:06 +00:00
7d4009cf8e
Added README.md and renamed LICENSE to all caps
Farhan Khan
2020-12-17 01:12:26 +00:00
dc55070065
Renaming the sample configuration file to avoid accidental commits
Farhan Khan
2020-12-16 19:10:54 +00:00
f8aa43ec4d
working through post and account verification
Farhan Khan
2020-12-16 06:21:18 +00:00
a50916eb36
using the same http client, adding it to running instances some web work lost track of what's going on and I'm not accountable to anyone...so screw it
Farhan Khan
2020-12-16 02:41:45 +00:00
3d196ccf41
Added multiple post handlers + exiting the main goroutine
Farhan Khan
2020-12-14 23:09:51 +00:00
8aff111336
intermediate parsing web received
Farhan Khan
2020-12-14 21:50:31 +00:00
4d5d3a8a2b
port http starting message
Farhan Khan
2020-12-13 23:33:38 +00:00
fd56285e71
Added capture type to engine
Farhan Khan
2020-12-13 08:26:50 +00:00
7286af9c79
updating client with capture type
Farhan Khan
2020-12-13 08:25:58 +00:00
01725debbb
moving startctl down
Farhan Khan
2020-12-13 07:39:35 +00:00
69d940f64b
Reusing oauth client information
Farhan Khan
2020-12-13 07:38:03 +00:00
9e2af34c15
building database connection string from config file
Farhan Khan
2020-12-12 05:50:43 +00:00
10013323ef
renaming stuff
Farhan Khan
2020-12-12 03:04:03 +00:00
f1e298e3af
Adding git ignore file
Farhan Khan
2020-12-12 02:58:49 +00:00
abc199b9ac
added streaming for mastodon streaming authentication does not refresh yet... oh, and migrated development from freebsd to linux
Farhan Khan
2020-12-11 17:20:44 +00:00
e68ef2a0f2
how did I not earlier commit the Makefile?
Farhan Khan
2020-12-08 12:34:37 -05:00
1d4d5dd78a
renamed main.go to fedilogue.go
Farhan Khan
2020-12-07 15:49:05 -05:00
a3644cce8f
changed url to uri
Farhan Khan
2020-12-07 15:47:44 -05:00
f7dd23bc55
normalizing additions + light reorg
Farhan Khan
2020-12-06 18:05:17 -05:00
7f293b9dd3
removed comments
Farhan Khan
2020-12-06 00:16:04 -05:00
52fd7bb7cb
adding oauth file, need to get better at this
Farhan Khan
2020-12-06 00:01:53 -05:00
0342844a52
partial implemented host banning
Farhan Khan
2020-12-05 23:59:32 -05:00
a4fa8a48c8
added authentication
Farhan Khan
2020-12-05 23:47:12 -05:00
f9a2e86627
added config option for tor
Farhan Khan
2020-12-04 14:19:26 -05:00
d474d98f84
Gets the settings, whoopsie on the order of commits
Farhan Khan
2020-12-03 17:26:38 -05:00
003de16e2b
checking for crawling boolean
Farhan Khan
2020-12-03 17:23:52 -05:00
d57dd47c56
config modification example
Farhan Khan
2020-12-02 21:18:36 -05:00
9106d6e22c
adding json configuration template
Farhan Khan
2020-12-02 18:56:25 -05:00
279e2e7441
minor stuff
Farhan Khan
2020-11-30 19:50:21 -05:00
911c30b409
more stability and retries changed min_id to since_id due to bug in mastodon
Farhan Khan
2020-11-28 11:42:31 -05:00
53279ff649
intermediate commit, node type determination, working on time format lots of crashing
Farhan Khan
2020-11-25 18:24:56 -05:00
a81c61d5dc
removed cruft
Farhan Khan
2020-11-24 21:19:39 -05:00