Commit graph

107 commits

Author SHA1 Message Date
コルセット姫@がんばらない
17124f3834 feat: Add media to Mastodon and Calckey post imports (#10496)
### What does this PR do?

Adding files fields in the export notes option, and corresponding import notes

Current the mastodon import does not import any attachments, this pr will use the "upload from url" feature to include medias if its a valid URL.

There are many way to convert the outbox.json file, can be simple as upload media_attachments to any web hosting and do string replace on the json file.

I also create a tool that upload the tar.gz file with auto convert and host the media as simplify the process at https://tempfile.moegirl.live

Detail example can be found at https://fedi.moegirl.live/notes/9h76gtqnp2gwl5dz

https://r2temp.moegirl.live/2023/7/15/15356683-050f-423a-b331-c9a05561f52a/shana-settings-_-meng-zhai-le-yuan-xyou-yu-ou-xiang-de-luo-ke-ke-wu-yan-moe-otaku-elysian-x-gloomily-idol-s-rococo-luncheon----mozilla-firefox-private-browsing-2023-07-15-18-36-37.mp4

Co-authored-by: CGsama <CGsama@outlook.com>
Reviewed-on: https://codeberg.org/calckey/calckey/pulls/10496
Co-authored-by: コルセット姫@がんばらない <cgsama@noreply.codeberg.org>
Co-committed-by: コルセット姫@がんばらない <cgsama@noreply.codeberg.org>
2023-07-15 22:56:09 +00:00
DVD
49d599144f Fix CW Import 2023-07-13 09:07:35 +08:00
ThatOneCalculator
5ad6dba98d perf: use fast-blurhash for blurhash decoding, up deps 2023-07-10 21:09:07 -07:00
PrivateGER
d9c027e286 Fix Postgres error loop when indexing notes 2023-07-10 23:44:31 +02:00
ThatOneCalculator
c4d4e850b2 chore: ⬆️ up prettier, pnpm 2023-07-05 18:28:27 -07:00
Namekuji
f13d8548d2 fix: do not deliver poll result if local-only 2023-07-04 01:59:43 -04:00
ThatOneCalculator
f99e8b3726 fix: immediately fail jobs with invalid signature 2023-07-03 10:29:52 -07:00
ThatOneCalculator
63b9a26621 adjust timeout for indexing notes 2023-07-02 00:47:55 -07:00
Namekuji
284c0c58c4 don't broadcast poll updates 2023-06-18 13:23:59 -04:00
Namekuji
91acab8860 skip if actor is not found or has been already deleted 2023-06-16 04:49:11 -04:00
Namekuji
40267c1116 fix inbox stall 2023-06-07 10:01:16 -04:00
PrivateGER
cc2f5b516f formatter 2023-06-01 21:38:37 +02:00
PrivateGER
2c2532b87d missing inversion + use proper emoji path 2023-06-01 21:30:23 +02:00
PrivateGER
e705b24005 me when I lack a brain 2023-06-01 21:12:03 +02:00
PrivateGER
6146ca270a more logging 2023-06-01 21:01:15 +02:00
PrivateGER
e05b0d0e97 Import emojis without meta.json 2023-06-01 20:53:13 +02:00
PrivateGER
51160a6a77 Delete deleted posts from Meilisearch 2023-05-29 18:33:57 +02:00
ThatOneCalculator
62835aa4a3 chore: formatting 2023-05-28 20:34:18 -07:00
PrivateGER
40ad37b870 Default meilisearch data response + linting + prettyfier 2023-05-28 00:27:16 +00:00
PrivateGER
2f20cf476b Lint + formatting 2023-05-28 00:27:16 +00:00
PrivateGER
f5aa0c86cf Load relationships to populate user fields 2023-05-28 00:27:16 +00:00
PrivateGER
206f7bfc09 Add high performance batch imports 2023-05-28 00:27:16 +00:00
April John
6f4465aa32 update post import 2023-05-24 20:22:16 +02:00
Namekuji
53a8e3ffc2 enhance: emoji width and height 2023-05-19 22:26:13 -04:00
Kainoa Kanter
d2106d4904 fix: dont show cheat sheet when not needed 2023-05-16 20:31:13 -07:00
April John
698a2c6912 Fix 2023-05-15 13:03:48 +00:00
April John
a2a9415839 Fix: add done statements 2023-05-15 12:25:54 +00:00
Kainoa Kanter
932929a4f1 chore: formatting 2023-05-14 15:20:26 -07:00
April John
6dc65e43af Feat: move post imports to new jobs 2023-05-14 20:20:31 +02:00
April John
6656c5f0cf Feat: move post imports to new jobs 2023-05-14 18:46:48 +02:00
April John
909b12dff2 reintroduce async into post import 2023-05-10 22:26:31 +02:00
Essem
5b37235a29 Switch to adm-zip for emoji pack handling
Unzipper seems to corrupt files in certain cases on Node 18/19.
See https://github.com/ZJONSSON/node-unzipper/issues/271
2023-05-09 19:19:15 -05:00
April John
3fda23c7dd Revert "fix: run post import async"
This reverts commit 3bedd1b34799d969cd218b4a1a67651532a204e6.
2023-05-08 22:48:12 +02:00
ThatOneCalculator
b92237c8d2 chore: formatting 2023-05-08 13:36:53 -07:00
April John
369fd4daea fix: run post import async 2023-05-08 22:30:40 +02:00
April John
3a8e6a19b2 import replies too 2023-05-08 22:09:42 +02:00
April John
77f74f51b3 try fetching replies 2023-05-08 22:09:42 +02:00
April John
13b5a0052e Feat: new internal post visibility "hidden" 2023-05-05 15:16:10 +02:00
ThatOneCalculator
19c4a59513 formatting 2023-03-30 19:10:03 -07:00
cutestnekoaqua
1b61d1a58d check for signatures while post import 2023-03-30 17:52:17 +02:00
cutestnekoaqua
cf5671cb05 nyaa 2023-03-30 17:31:29 +02:00
Cleo
7a31740e20 remove debug msgs 2023-03-30 09:30:38 +00:00
Cleo
fcbdde47d0 Yeet mentions 2023-03-30 08:40:20 +00:00
cutestnekoaqua
4df655126d get post details from object 2023-03-29 21:38:52 +02:00
cutestnekoaqua
db392efae8 catch text parsing error 2023-03-29 21:25:28 +02:00
cutestnekoaqua
41c7dfe208 add info 2023-03-29 21:16:45 +02:00
cutestnekoaqua
da322a086b catch errors 2023-03-29 21:15:06 +02:00
cutestnekoaqua
d640af9714 handle mastodon style posts 2023-03-29 21:10:01 +02:00
cutestnekoaqua
365d9bc40a dont ping users 2023-03-29 20:01:56 +02:00
cutestnekoaqua
dfb6178ddf actually increase line num 2023-03-29 19:37:31 +02:00