Commit Graph

  • 4c41389e9b Detect CAR files with zero blocks and handle them accordingly Max Ignatenko 2024-03-17 17:42:09 +0000
  • 638bdcf515 Add a check for zero bytes fetched from PDS Max Ignatenko 2024-03-17 17:14:04 +0000
  • bc895b6425 Update dash. mathan 2024-03-17 08:52:28 -0700
  • 553894dc6a Switch to partial repo fetches Max Ignatenko 2024-03-17 15:34:26 +0000
  • 346aa7c539 Document consistency model for repo data Max Ignatenko 2024-03-17 15:21:54 +0000
  • 0cc11e75b2 Avoid updating records when then didn't actually change Max Ignatenko 2024-03-17 14:16:25 +0000
  • 9c51a4621f Start recording last rev for each repo Max Ignatenko 2024-03-13 22:27:04 +0000
  • 85259baa1c Update dash. mathan 2024-03-13 14:21:05 -0700
  • 9d429be424 Update dash. mathan 2024-03-13 11:33:13 -0700
  • 76a50524a0 Give postgres plenty of time to shutdown cleanly Max Ignatenko 2024-03-13 18:12:55 +0000
  • 44d2b25951 Fix nil pointer deref Max Ignatenko 2024-03-13 16:38:58 +0000
  • e150f1da90 Populate FirstRevSinceReset if empty Max Ignatenko 2024-03-13 11:32:23 +0000
  • 87d510e67a Check repos against PDS cursor resets, instead of waiting for a first new even for them on firehose Max Ignatenko 2024-03-13 10:39:41 +0000
  • 57aa4731e5 Properly update repo consistency metadata Max Ignatenko 2024-03-13 10:36:41 +0000
  • 080cbb8b12 Bump WAL size limit to 8GB Max Ignatenko 2024-03-13 09:51:34 +0000
  • 75386c0235 Add export_blocks view. Add iexport + audit table. mathan 2024-03-12 21:15:23 -0700
  • e1ec43def7 Add count listitems. mathan 2024-03-06 18:58:08 -0800
  • dc80cd95ad Update stats dash. mathan 2024-03-05 14:17:46 -0800
  • 209813bd87 Update dash + csv_export. mathan 2024-02-28 10:37:18 -0800
  • 439518fda1 Update dash. mathan 2024-02-26 12:57:05 -0800
  • 3a2c6a6f5b Add PDS URL to error message Max Ignatenko 2024-02-26 07:27:15 +0000
  • efc106453f Fixed NPE, fixed failed lister migration. mathan 2024-02-25 20:51:10 -0800
  • 067be8ec83 Update dash. Add smartmon to readme. mathan 2024-02-25 18:04:06 -0800
  • 95ca4f0eef Use materialized views for export to csv Max Ignatenko 2024-02-25 22:05:07 +0000
  • 2172651870 Merge branch 'main' of github.com:uabluerail/indexer mathan 2024-02-25 09:05:50 -0800
  • 95abc3691f Update dash, added smartmon mathan 2024-02-25 09:05:48 -0800
  • b4a0c268d6 Bump max_parallel_workers_per_gather to 8 Max Ignatenko 2024-02-25 11:35:10 +0000
  • fa592c2b34 Add a migration to enable compression on already existing tables Max Ignatenko 2024-02-25 11:24:40 +0000
  • 3432121be2 Give postgres a bit more RAM Max Ignatenko 2024-02-25 11:03:48 +0000
  • 0b6e326762 Set an explicit default for Deleted Max Ignatenko 2024-02-25 10:30:49 +0000
  • fdace096fb Add fast compression to save a bit on IO bandwidth Max Ignatenko 2024-02-25 10:29:08 +0000
  • b9b3159828 Update dashboards mathan 2024-02-24 20:01:16 -0800
  • fa87ce6105 Update dashboards mathan 2024-02-23 18:35:38 -0800
  • a5ad0743ee Update stats dash mathan 2024-02-23 14:22:39 -0800
  • d18867f495 Update stats dash. mathan 2024-02-23 13:37:59 -0800
  • e98c4f4c63 Add stats dash. Update indexer dash mathan 2024-02-23 12:48:52 -0800
  • bd4e74d36f Update dash mathan 2024-02-23 12:33:53 -0800
  • 9fb8fc3dce Update dash mathan 2024-02-23 11:44:06 -0800
  • 69c5c6b7ad Add ignore to override. mathan 2024-02-23 09:26:11 -0800
  • ddde20a014 Fix comparison with NULL when listing PDSs Max Ignatenko 2024-02-23 15:22:22 +0000
  • 6bb2e5a1de Fix path to sql file Max Ignatenko 2024-02-23 11:28:24 +0000
  • 13b911615d Add sqldu Max Ignatenko 2024-02-23 11:26:06 +0000
  • ea3e11e495 Add sqltop Max Ignatenko 2024-02-23 11:19:13 +0000
  • 0f191b2609 Increment exported vars for every language in a post Max Ignatenko 2024-02-23 11:12:39 +0000
  • 297e9e0347 Scripts for benchmarking partitioning Max Ignatenko 2024-02-23 11:10:23 +0000
  • 78a17bf238 Merge branch 'main' of github.com:uabluerail/indexer mathan 2024-02-22 18:56:19 -0800
  • db425b1d5f Fix view in migration. Add by lang metric to consumer. mathan 2024-02-22 18:54:29 -0800
  • a20ddf0717 Fix the fucking regexp Max Ignatenko 2024-02-22 18:15:59 +0000
  • a28199fb92 Handle the new #identity message Max Ignatenko 2024-02-22 12:16:24 +0000
  • 8f32c494f7 Add whitelist for PDS hosts and update repo PDS pointer on appropriate occasions Max Ignatenko 2024-02-22 12:05:44 +0000
  • 600dac7694 Added metric for posts by lang (olivčykom). mathan 2024-02-21 18:09:12 -0800
  • 3c632ebe97 Update csv_export mathan 2024-02-21 13:40:18 -0800
  • 9d4bb721cf Fix makefile mathan 2024-02-21 10:22:00 -0800
  • 9a520c186e Fix dids dumping Max Ignatenko 2024-02-21 17:40:23 +0000
  • 59e343355e Add a few indexes and views Max Ignatenko 2024-02-21 10:36:55 +0000
  • 0898658acb Split out list-related records into separate partitions Max Ignatenko 2024-02-21 10:08:50 +0000
  • 765b7b792b Set default PDS ID to unknown Max Ignatenko 2024-02-21 09:39:26 +0000
  • 67a8ab5aff Downgrade intermediate resolver error to trace Max Ignatenko 2024-02-21 09:36:09 +0000
  • b2003530ba Handle repos with unknown PDS Max Ignatenko 2024-02-21 09:35:25 +0000
  • fa8b67bcc9 Fix tar output filename Max Ignatenko 2024-02-21 09:16:19 +0000
  • d6dff5a9a0 Fix csv_export and Makefile mathan 2024-02-20 22:08:14 -0800
  • 0e01664cfb Added example content jsons. Updated README.md mathan 2024-02-20 22:00:26 -0800
  • 3b77099bec Add daemon csv export to Makefile mathan 2024-02-20 20:43:50 -0800
  • fba373f835 Add db-migrations folder. Add future language metrics to query-exporter. mathan 2024-02-20 15:45:23 -0800
  • da86ee3a1c JSON indexes for posts Max Ignatenko 2024-02-20 22:58:29 +0000
  • de4017bc83 Add delimiters, fixed likes file name. mathan 2024-02-20 11:07:47 -0800
  • b7458a18f4 Export the list of repos from indexer Max Ignatenko 2024-02-20 15:36:52 +0000
  • d9147940aa Bump connection limit for postgres from 100 to 1000 Max Ignatenko 2024-02-20 15:35:59 +0000
  • 876b454916 Patch through POSTRES_PASSWORD and export the number of failed repos Max Ignatenko 2024-02-19 19:53:58 +0000
  • b809b8b983 Rename dump.sh mathan 2024-02-19 15:11:09 -0800
  • 95c24ff0c5 Add maitenance queries mathan 2024-02-19 11:22:51 -0800
  • 4ef6abbfc4 Update readme mathan 2024-02-19 11:20:34 -0800
  • f5a8be6ebb Script for dumping data into .csv Max Ignatenko 2024-02-19 19:06:16 +0000
  • 6b400b1f43 Add a script for breaking up records table into partitions Max Ignatenko 2024-02-19 19:05:50 +0000
  • 8b86f8f929 Merge branch 'main' of github.com:uabluerail/indexer mathan 2024-02-19 10:59:24 -0800
  • 4b06e54cdc Update dash mathan 2024-02-19 10:59:23 -0800
  • a0934c360e fix division by zero Max Ignatenko 2024-02-19 18:34:26 +0000
  • e8ca4e808e Move query-exporter to docker. Update dash. mathan 2024-02-19 10:18:40 -0800
  • 758c5fe5e6 Add quick&dirty quarantine logic for bad records Max Ignatenko 2024-02-19 17:06:19 +0000
  • 4418d76bb2 Dash update mathan 2024-02-18 16:58:29 -0800
  • 4f259dccbf Move panels mathan 2024-02-18 16:40:44 -0800
  • 3e34e4ca02 Add s/s graph mathan 2024-02-18 16:39:29 -0800
  • a73c638116 Fixed disk space on dash. Avail -> free. mathan 2024-02-18 13:33:57 -0800
  • 252501d57a Update dashboard mathan 2024-02-18 13:18:39 -0800
  • 655a3ca758 Merge branch 'main' of github.com:uabluerail/indexer mathan 2024-02-18 12:35:54 -0800
  • 07f9c5d3e9 Fix readme mathan 2024-02-18 12:33:00 -0800
  • e92be0a999 make psql Max Ignatenko 2024-02-18 20:29:14 +0000
  • 82197c63ff Error out after creating .env Max Ignatenko 2024-02-18 20:27:17 +0000
  • 495fbaab6a Add a simple Makefile Max Ignatenko 2024-02-18 20:23:54 +0000
  • c413ab41c2 Add query exporter mathan 2024-02-18 12:00:39 -0800
  • 5a24b6eff6 Update docker-compose mathan 2024-02-18 10:35:20 -0800
  • 5e87198276 Move prometheus + grafana to metrics. Edit readme mathan 2024-02-18 10:27:51 -0800
  • 8ad97d6752 Update readmes mathan 2024-02-18 09:56:00 -0800
  • c391db0a42 Update mathan 2024-02-18 09:39:38 -0800
  • 1d25842b78 Add few more metrics Max Ignatenko 2024-02-18 17:23:54 +0000
  • 334af033b8 Add prometheus + grafana mathan 2024-02-18 07:37:50 -0800
  • 8b89a1544e Log failed resolve attempts in fallbackResolver Max Ignatenko 2024-02-18 15:34:14 +0000
  • 1038ca3bea Add AtRev column to only overwrite records with a newer version Max Ignatenko 2024-02-17 14:29:45 +0000
  • 1d3c6edf0a Fix panic on closing already closed channel Max Ignatenko 2024-02-17 12:26:57 +0000
  • 04d521f58c Increase request timeout, default 30 seconds are not enough to download some repos Max Ignatenko 2024-02-16 16:47:36 +0000