astra
/
plc-mirror
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update csv_export

main
mathan 2024-02-21 13:40:18 -08:00
parent 9d4bb721cf
commit 3c632ebe97
4 changed files with 5 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -2,4 +2,4 @@ go.work*
.env
*.csv
*.out
csv_export.gz
csv_export.tgz

2
Makefile
View File

@ -58,6 +58,6 @@ kill-csv-export:
@kill -9 `pgrep csv_export.sh`
csv-compress:
@tar cvzf csv_export.tgz handles.csv post_counts.csv follows.csv like_counts.csv
@tar cvzf csv_export.tgz dids.csv handles.csv post_counts.csv follows.csv like_counts.csv
# ---------------------------- CSV Export ----------------------------

2
csv_export.sh
View File

@ -52,7 +52,7 @@ echo "Done: $(ls -lh post_counts.csv)"
# ------------------------------ REPLIES ----------------------------------
# ------------------------------ HANDLES ----------------------------------
# ------------------------------ DIDS + HANDLES ----------------------------------
dids_query="$(cat <<- EOF
insert into repos (did)

5
explore/README.md
View File

@ -38,12 +38,11 @@ Get list blocks
non-partitioned (very slow)
```
select count(*) from (select distinct repo from records where collection in ('app.bsky.graph.listblock') and content['subject']::tex
t like '"at://did:plc:bmjomljebcsuxolnygfgqtap/%');
select count(*) from (select distinct repo from records where collection in ('app.bsky.graph.listblock') and deleted=false and content['subject']::text like '"at://did:plc:bmjomljebcsuxolnygfgqtap/%');
```
partitioned (slow)
`select count(*) from (select distinct repo from records_listblock where content['subject']::text like '"at://did:plc:bmjomljebcsuxolnygfgqtap/%');`
`select count(*) from (select distinct repo from records_listblock where deleted=false and content['subject']::text like '"at://did:plc:bmjomljebcsuxolnygfgqtap/%');`
Count all records