astra
/
plc-mirror
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Commit Graph

  • b00ce6d799 Merge remote-tracking branch 'bsky-mirror/main' main Astra 2024-11-14 18:07:10 +0000
  • bad22e6422 fix: s/LastSuccess/LastCompletion/ and use it as orignally intended Max Ignatenko 2024-11-14 12:44:42 +0000
  • 04ab6167c7 fix: use LastRecordTimestamp in serve() instead of LastSuccess Max Ignatenko 2024-11-14 12:36:39 +0000
  • b26866fe0c fix: update lastSuccessTimestamp after each request too Max Ignatenko 2024-11-14 12:34:13 +0000
  • 2c6a7201ef fix: actually store the queried timestamp in the struct Max Ignatenko 2024-11-14 12:17:01 +0000
  • 04f2f80a06 fix: keep the timestamp of last record in memory instead of querying the DB every time Max Ignatenko 2024-11-13 18:56:22 +0000
  • 80128ebcff fix: change lower bound 1 -> 0.1 Max Ignatenko 2024-10-27 12:23:59 +0000
  • f379f6b4b4 fix: tweak histogram buckets Max Ignatenko 2024-10-27 12:21:59 +0000
  • d0fd66f83f feature: a few exported metrics Max Ignatenko 2024-10-26 21:31:44 +0100
  • 61f751b59a update files Astra 2024-10-26 02:59:57 +0100
  • ca4d5709e8 Update README.md Astra 2024-10-26 03:54:30 +0200
  • bb036dc6d8 Update README.md Astra 2024-10-26 03:53:15 +0200
  • add8ffc1c4 Update cmd/plc-mirror/Dockerfile Astra 2024-10-26 03:50:41 +0200
  • 90370ef60b Update cmd/plc-mirror/serve.go Astra 2024-10-25 23:58:58 +0200
  • 49182413f0 Update cmd/plc-mirror/mirror.go Astra 2024-10-25 23:58:45 +0200
  • 4432fcea80 Update cmd/plc-mirror/main.go Astra 2024-10-25 23:58:25 +0200
  • 39c3c9cd5a Update go.mod Astra 2024-10-25 23:56:56 +0200
  • 9f29e6cc24 Rename module and run `go mod tidy` Max Ignatenko 2024-09-25 22:05:40 +0100
  • 9b877c1524 Remove everything that's not needed for PLC mirror Max Ignatenko 2024-09-25 22:00:00 +0100
  • 56727bbe11 fix: properly install curl Max Ignatenko 2024-09-07 19:36:48 +0100
  • 87a66bd269 fix: remove deprecated field Max Ignatenko 2024-09-07 13:52:39 +0100
  • db74fac7d0 feature: add plc-mirror healthcheck Max Ignatenko 2024-09-07 13:48:19 +0100
  • a41f6aa75c wip: try partitioning by date Max Ignatenko 2024-09-07 13:29:50 +0100
  • 19a21ad3a9 feature: skip unnecessary recompilation in some cases Max Ignatenko 2024-09-07 11:47:15 +0100
  • a8bcde76e2 feature: move schema updates into a separate oneshot service Max Ignatenko 2024-09-07 11:18:31 +0100
  • de51bd9015 Read handles from the new plc mirror Max Ignatenko 2024-06-24 16:31:50 +0100
  • 9cc2572666 Filter out dead dids on postgres level. mathan 2024-06-03 14:53:43 -0700
  • 4af4f48a1e Update dash. mathan 2024-05-30 07:55:35 -0700
  • d57f4750d9 Add listblock query. mathan 2024-05-27 10:38:39 -0700
  • 8de7cb87cb Fix regexp to include bsky.social in 1st-party PDS list Max Ignatenko 2024-05-21 17:05:14 +0100
  • 51986e34c6 Update dashboard Max Ignatenko 2024-05-21 17:02:36 +0100
  • e6176a2217 Add exponential backoff to firehose connection attempts and export connection status directly Max Ignatenko 2024-05-21 11:43:35 +0100
  • e986d370ec go get -u ./... Max Ignatenko 2024-05-21 09:47:36 +0100
  • 722952af34 go mod tidy Max Ignatenko 2024-05-21 09:44:21 +0100
  • 9283a841b4 Fix for atproto.brid.gy being treated as not whitelisted due to not matching "https://*" glob pattern Max Ignatenko 2024-05-21 08:58:37 +0100
  • a9076c44c0 Ladder fixed. mathan 2024-05-16 22:17:17 -0700
  • 64b7533315 remove the latest migration until further notice. mathan 2024-05-16 17:29:07 -0700
  • 6fe357b54a Fix decay values. mathan 2024-05-16 08:32:16 -0700
  • ca7d5956eb Cutoff at 240 days. mathan 2024-05-15 20:56:06 -0700
  • 621ce539cf Undo fix. mathan 2024-05-15 13:28:21 -0700
  • 8daf08d57c Fix optout header. mathan 2024-05-15 11:51:27 -0700
  • fb1048f3a6 Add temporal ladder export. Fix optout export typo. mathan 2024-05-15 10:40:23 -0700
  • 59922eb54c Add monthly export. mathan 2024-05-14 19:26:00 -0700
  • a568e16624 Merge branch 'main' of github.com:uabluerail/indexer mathan 2024-05-06 17:04:58 -0700
  • 53db0fc5e7 Added optouts to export. mathan 2024-05-06 17:04:57 -0700
  • 9c375a3458 go get -u golang.org/x/net Max Ignatenko 2024-05-06 22:49:15 +0100
  • 1d68d70a41 Avoid lister having stuck on a single broken PDS Max Ignatenko 2024-05-06 21:17:42 +0100
  • f315059994 Avoid querying too few repos from each single PDS Max Ignatenko 2024-05-06 21:14:18 +0100
  • 1201ee3868 Update dash. mathan 2024-04-25 08:35:54 -0700
  • a17250788d Update dash. mathan 2024-04-16 14:17:18 -0700
  • 6997d51c07 Split firehose graphs between 1st- and 3rd-party PDSs Max Ignatenko 2024-04-16 17:01:31 +0100
  • 4c3b1c7acc Make hostname abbreviations uniform Max Ignatenko 2024-04-16 16:46:02 +0100
  • 5cb8befb0d Add my dashboard as a default Max Ignatenko 2024-04-15 22:00:21 +0100
  • 78b4f54527 Add wiring for maintaining a dashboard Max Ignatenko 2024-04-15 21:58:33 +0100
  • c364822818 Tweak rate limit a little bit Max Ignatenko 2024-04-14 12:58:42 +0100
  • bc24c23afc Switch to batch inserts Max Ignatenko 2024-04-14 12:45:05 +0100
  • c204e48de1 Add missing word Max Ignatenko 2024-04-13 22:48:34 +0100
  • 2271f4da1f Update readme Max Ignatenko 2024-04-13 21:03:34 +0100
  • 9af4101a7a Update readme Max Ignatenko 2024-04-13 20:59:40 +0100
  • 4c29c3f179 Add license Max Ignatenko 2024-04-13 20:41:56 +0100
  • fa88777ecd Update readme Max Ignatenko 2024-04-13 20:38:58 +0100
  • 60c89c8796 Fix Makefile Max Ignatenko 2024-04-13 20:24:01 +0100
  • c5f3a55ac8 Add a target for waiting until PLC mirror catches up Max Ignatenko 2024-04-13 19:42:36 +0100
  • c6fe0c2350 Make the address for exporting metrics configurable Max Ignatenko 2024-04-13 19:01:51 +0100
  • c09052eb4c Tweak example configs Max Ignatenko 2024-04-13 18:56:14 +0100
  • a746fd7120 Add deps on .env in couple more places Max Ignatenko 2024-04-13 18:50:06 +0100
  • 81152c0bfc Add block subjects to exported dids Max Ignatenko 2024-04-13 18:45:04 +0100
  • 7af2a3f8be Remove compression from content columns Max Ignatenko 2024-04-13 18:42:26 +0100
  • fe19806af2 Add code missing in the previous commit Max Ignatenko 2024-04-13 18:37:26 +0100
  • d6b5850827 Add plc mirror Max Ignatenko 2024-04-13 16:45:02 +0100
  • c17730c11f Drop the hack for detecting cursor reset from non-compliant servers Max Ignatenko 2024-04-10 23:18:16 +0100
  • 4b40c5919b Don't update LastFirehoseRev more than once a day Max Ignatenko 2024-04-07 15:06:13 +0100
  • 581cf8c720 go mod tidy Max Ignatenko 2024-04-07 14:55:02 +0100
  • 99c0b08eab Write more words Max Ignatenko 2024-04-07 14:53:22 +0100
  • 52dd38f11b Don't attribute context cancellation to firehose record content Max Ignatenko 2024-04-06 22:14:58 +0100
  • ecf2fc57d8 Disconnect from firehoses in parallel when shutting down Max Ignatenko 2024-04-06 21:59:21 +0100
  • 1358bc3f08 Export a counter of firehose connection errors Max Ignatenko 2024-04-06 21:55:36 +0100
  • ff0ea08296 Implement commit signature validation Max Ignatenko 2024-04-06 21:50:32 +0100
  • f9dde4db39 Update dash. mathan 2024-04-03 12:14:49 -0700
  • 9cfc23b981 Update dash. mathan 2024-04-03 12:12:28 -0700
  • 89d3a264c9 Export the number of bad firehose records for each PDS Max Ignatenko 2024-03-29 12:02:05 +0000
  • 7c09c37a51 Pass the correct context to Consumer.Start so that it will actually stop when singalled Max Ignatenko 2024-03-29 10:15:48 +0000
  • 567885af60 Fix grafana not restarting. mathan 2024-03-28 14:57:20 -0700
  • 1abe505ef9 Add support for discovering new PDSs from relays Max Ignatenko 2024-03-28 20:55:02 +0000
  • c919050833 Keep the set of running consumers up to date Max Ignatenko 2024-03-28 20:02:48 +0000
  • 337f3ef2b8 Delete commented out code Max Ignatenko 2024-03-28 18:46:39 +0000
  • 3da7228f35 Wait for postgres to become ready before starting other containers Max Ignatenko 2024-03-28 16:16:38 +0000
  • fc5307a971 Add a bit more logging for cursor values Max Ignatenko 2024-03-28 16:00:19 +0000
  • 5d3d562ecc Stop the ticker when goroutine exits Max Ignatenko 2024-03-28 15:31:17 +0000
  • 4556d6738a Update makefile. mathan 2024-03-27 19:59:24 -0700
  • 3aca467516 Update iexport. mathan 2024-03-27 16:18:18 -0700
  • ffa2faa420 Properly escape null character in the consumer too Max Ignatenko 2024-03-24 12:43:52 +0000
  • 693ae1ba0a Bump the limit on stored bad records to 500 Max Ignatenko 2024-03-24 12:32:13 +0000
  • d189912bcf Update dash. Add alerting webhook. mathan 2024-03-20 07:18:15 -0700
  • bae23a62d0 Don't try to upsert zero records Max Ignatenko 2024-03-20 13:30:51 +0000
  • 542be03088 Add all PDSs to whitelist Max Ignatenko 2024-03-18 16:24:29 +0000
  • dc17238a93 Update dash mathan 2024-03-18 18:01:47 -0700
  • c3bd6d177a Expand some terms used Max Ignatenko 2024-03-17 22:03:16 +0000
  • e8c816a3a3 Update FirstCursorSinceReset even if we received zero new records Max Ignatenko 2024-03-17 21:22:34 +0000
  • 328a676e2a Fix potential infinite loop for inactive repos after a cursor reset Max Ignatenko 2024-03-17 19:36:12 +0000