Compare commits

..

84 Commits

Author SHA1 Message Date
Oleksii Holub 5632d6c386 HTML-encode markdown content when formatting is disabled (#1545) 2026-06-04 13:09:03 +03:00
Oleksii Holub f4d1e630f7 Implement missing HTML encoding for several attribute injections (#1544) 2026-06-04 12:38:09 +03:00
Colton 799dd1ad81 Stream asset downloads to avoid OutOfMemoryException on large files (#1540)
* Stream asset downloads to avoid OutOfMemoryException on large files

ExportAssetDownloader used HttpClient.GetAsync(url, ct), which defaults to
HttpCompletionOption.ResponseContentRead and buffers the entire response body
into memory before returning. Downloading a large attachment therefore tried to
grow a single in-memory buffer to the full file size and threw
OutOfMemoryException (HttpContent.LoadIntoBufferAsync -> GrowAndWrite).

Pass HttpCompletionOption.ResponseHeadersRead so the body is streamed straight
to disk via CopyToAsync in bounded chunks, matching how DiscordClient already
issues its requests.

* Update ExportAssetDownloader.cs

---------

Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2026-06-03 15:47:43 +03:00
dependabot[bot] 64a4ba296e Bump the actions group with 4 updates (#1541)
Signed-off-by: dependabot[bot] <support@github.com>
2026-06-02 19:41:24 +03:00
dependabot[bot] f0e99765f7 Bump the nuget group with 13 updates (#1542)
Signed-off-by: dependabot[bot] <support@github.com>
2026-06-02 19:40:41 +03:00
tyrrrz 45629c9d7f Refactor message boxes 2026-05-26 15:00:42 +03:00
Copilot 2aec84884b Extract app shutdown fallback into App.Shutdown (#1539)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-05-25 19:10:19 +03:00
tyrrrz 6258394fc0 Remove lang version from build props 2026-05-03 16:01:32 +03:00
tyrrrz e6472543e0 Refactor fetching Avalonia TopLevel 2026-05-02 11:30:14 +03:00
dependabot[bot] ee163d6de7 Bump actions/upload-artifact from 7.0.0 to 7.0.1 in the actions group (#1528)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-01 16:31:01 +03:00
tyrrrz 5838c0546e Update for Avalonia 12 2026-05-01 16:30:11 +03:00
dependabot[bot] 07a02773be Bump the nuget group with 15 updates (#1529)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-01 16:05:31 +03:00
tyrrrz 0a623b24d1 Refactor GUI spacing 2026-04-22 17:18:28 +03:00
tyrrrz da4f5ea74a Use PowerKit's Uri.TryGetFilaName() 2026-04-20 22:22:11 +03:00
tyrrrz 34b5ccc75b Remove BOM 2026-04-19 23:13:12 +03:00
Copilot 7456f0fe3a Add PowerKit and replace custom utility extensions (#1525)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-19 23:10:45 +03:00
tyrrrz 757daa7a60 Forward slashes 2026-04-19 18:13:26 +03:00
tyrrrz 9590d4461a Standardize repo structure 2026-04-19 16:24:23 +03:00
tyrrrz a27fc3e72e Update Avalonia 2026-04-17 15:02:28 +03:00
Oleksii Holub f10bba63e6 Add warning against user account automation 2026-04-14 22:52:10 +03:00
tyrrrz b76b42549b Fix security issues 2026-04-14 13:23:38 +03:00
tyrrrz ae70fe9741 Move theme initialization to the correct place 2026-04-07 17:54:05 +03:00
tyrrrz af3ec8c299 Use CliFx v3 stable 2026-04-03 17:27:30 +03:00
tyrrrz f1673cde77 Use the Loaded event in view init 2026-04-03 11:37:44 +03:00
tyrrrz ed33d19bc0 Fix settings getting corrupted 2026-04-03 11:20:54 +03:00
tyrrrz 80e9ad75ba Clean up 2026-04-03 00:35:09 +03:00
tyrrrz 295f4cf9a2 Improve WatchProperty 2026-04-03 00:24:27 +03:00
tyrrrz 1f189e5206 Rename view model manager methods 2026-04-02 14:50:11 +03:00
tyrrrz df683b3b41 Remove unneeded usings 2026-04-02 14:36:17 +03:00
tyrrrz d467948d0b Only execute commands from code if allowed 2026-04-02 14:36:06 +03:00
tyrrrz f6166764e9 Automate view initialization 2026-04-02 14:35:17 +03:00
dependabot[bot] 7ee2763d4b Bump the nuget group with 17 updates (#1520)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-01 15:03:58 +03:00
dependabot[bot] eb99ab29ed Bump the actions group with 5 updates (#1519)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-01 15:02:33 +03:00
tyrrrz 5c61306231 Update favicon 2026-03-31 23:47:26 +03:00
tyrrrz 2e4601d378 Emit source-generated files for easier debugging 2026-03-30 21:36:11 +03:00
Oleksii Holub cc703cc860 Migrate to CliFx v3 (#1516) 2026-03-26 23:53:28 +02:00
Oleksii Holub 661f82c478 Add Nix download for GUI 2026-03-24 00:51:42 +02:00
tyrrrz b9f19f7003 Fix shebang, rename script 2026-03-23 23:34:57 +02:00
Copilot e794d7c833 Replace Publish-MacOSBundle.ps1 with a .NET 10 single-file C# script (#1515)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-03-23 23:18:46 +02:00
Oleksii Holub 5a26ba0b5a Update WinGet links in Readme 2026-03-21 23:55:12 +02:00
tyrrrz 1181a35f08 Variable names consistency 2026-03-21 20:27:05 +02:00
Copilot 4a62f9c34f Add DISCORDCHATEXPORTER_ALLOW_AUTO_UPDATE env var to suppress auto-update in GUI (#1514)
* Add DISCORDCHATEXPORTER_DISABLE_UPDATE env var to disable auto-update in GUI

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Tyrrrz/DiscordChatExporter/sessions/158dba86-9958-4f57-ab22-174e0606b42f

* Rename IsAutoUpdateDisabled to IsAutoUpdateAllowed and env var to DISCORDCHATEXPORTER_ALLOW_AUTO_UPDATE

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Tyrrrz/DiscordChatExporter/sessions/aa3944a4-e542-4770-8d17-2acb2c958ce9

* Refine IsAutoUpdateAllowed: group in (), invert, rename v to env, only accept false

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Tyrrrz/DiscordChatExporter/sessions/1467eee2-3069-4f0b-abdf-b1dc65e00a9a

* Fix CSharpier formatting: move is { } env to indented new line

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Tyrrrz/DiscordChatExporter/sessions/7933cfe4-ecad-4697-8b9c-ee3991aa147e

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-03-21 20:25:53 +02:00
Oleksii Holub b6e1f92989 Add the GUI flavor of the AUR package to readme 2026-03-21 17:14:45 +02:00
Oleksii Holub 0d3d2ddcf0 Add more package manager install options
Added community-maintained package installation instructions for Scoop and WinGet, and clarified notes regarding MacOS and community-maintained packages.
2026-03-21 17:03:57 +02:00
billA 9afecd47bd Update token extraction script in docs (#1511) 2026-03-16 15:24:54 +02:00
Tyrrrz 5cec162b4e More consistent message box 2026-03-12 18:54:44 +02:00
Tyrrrz 6647f90ced Update default branch ref 2026-03-04 20:23:55 +02:00
Copilot b9a3001c11 Add DISCORDCHATEXPORTER_SETTINGS_PATH env var override for settings file path (#1500)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-03-02 20:06:20 +02:00
Copilot dadfe5b8ef Guard App.Dispose() against double-invocation and late shutdown (#1499)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-03-02 18:42:35 +02:00
dependabot[bot] 4f29fa63d0 Bump the actions group with 2 updates (#1495)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-01 15:53:28 +02:00
dependabot[bot] 54b0174bde Bump the nuget group with 15 updates (#1496)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-01 15:53:11 +02:00
Tyrrrz 3be5bc6a21 Remove dashes from OutputPathTooltip 2026-02-27 19:14:30 +02:00
Tyrrrz c9f152773a Clean up last PR 2026-02-27 19:12:27 +02:00
Copilot 5e39611395 Double-clicking a channel auto-opens export setup (#1493)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-27 19:10:19 +02:00
Tyrrrz 40491bdc0f Fix text misalignment on export setup 2026-02-27 19:05:16 +02:00
Tyrrrz 03dd272eaf Fix tests harder this time 2026-02-27 18:59:52 +02:00
Tyrrrz 98e294f7e4 Fix tests, clean up 2026-02-27 18:36:16 +02:00
Tyrrrz b22e786449 Fix the shrug emoji not rendering correctly 2026-02-27 18:35:53 +02:00
Tyrrrz 951c1e4613 Make Run's BaselineAlignment set globally 2026-02-27 18:31:40 +02:00
Tyrrrz 675d910ea3 Formatting 2026-02-27 17:56:29 +02:00
Tyrrrz 28f26e45fb Add tests for forwarded messages 2026-02-27 17:52:26 +02:00
Tyrrrz 4674c517e3 Refactor 2026-02-27 17:39:36 +02:00
Tyrrrz b660edfe78 Clean up after last PR 2026-02-27 17:30:06 +02:00
Tyrrrz 532470e3db Fix format 2026-02-27 15:42:47 +02:00
farley128 b533e08d74 Implement exporting for forwarded messages (#1451)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2026-02-27 15:41:31 +02:00
Tyrrrz d490fd0a38 Improve, clean up encryption 2026-02-27 14:57:15 +02:00
Copilot 21c2398e2a Cleanup: fix filename typo, replace AsyncNonKeyedLocker, drop WithIndex(), use ProcessStartInfo ctor arg (#1492)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-27 14:42:07 +02:00
Tyrrrz 7ff2347df1 Clean up 2026-02-27 14:32:27 +02:00
Tyrrrz 3391e755fc Fix markdown not parsing for Persist Token tooltip 2026-02-27 14:19:03 +02:00
Tyrrrz f0e9d3be1b Reorganize files 2026-02-27 14:14:43 +02:00
Copilot eef0fc742d Encrypt Discord token at rest in settings file (machine-bound) (#1491)
* Initial plan

* Add token encryption when saving/loading settings

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Apply suggestion from @Tyrrrz

* Apply suggestion from @Tyrrrz

* Bind token encryption key to machine identity

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Switch to AES-GCM, hex encoding, and GetBytes/Fill improvements

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Address all review feedback: salt injection, code style, localization formatting

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Address latest review: ThisAssembly.Project, EnvironmentExtensions, inline Lazy, renames, localization wording

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Address latest review: layout comment, cipherSource, else block, MachineName fallback, csproj ordering

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Apply suggestion from @Tyrrrz

* Rename GetMachineId→TryGetMachineId, refactor Write to use single array with FillBytes

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Rename cipherSource→cipher in Read(), tokenBytes→tokenData in Write(), update layout comments

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Add cipherSource variable in Write(), update layout comment with size annotation

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Fix CSharpier formatting: inline multiline string assignments and reformat exception filter

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Quote EncryptionSalt argument to handle single quotes in secret value

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Revert double-quote fix on EncryptionSalt argument

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Apply suggestion from @Tyrrrz

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-27 14:01:25 +02:00
Tyrrrz 2e47c73388 Use range operator for substring extraction in StringSegment.ToString() 2026-02-27 11:30:09 +02:00
Tyrrrz 2d49146c74 Rename transformFallback parameter to fallbackTransform for clarity 2026-02-27 10:24:39 +02:00
Copilot c59374598a Migrate to Centralized NuGet Package Management (CPM) (#1489)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-26 21:29:46 +02:00
Tyrrrz 01d0203445 Clean up 2026-02-26 21:24:20 +02:00
Copilot c4bfb3424e Add option to reverse message order in exports (#1487)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-26 21:14:57 +02:00
Copilot 522caba420 Fix standard emoji not rendering as Discord-style images (#1488)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-26 17:20:04 +02:00
Copilot ad1170b42e Fix duplicate threads causing a crash when exporting in parallel (#1486)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-26 17:19:07 +02:00
Copilot d2c0e2169b Increase media asset URL hash length to eliminate collision risk with --reuse-media (#1485)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-26 16:35:18 +02:00
Tyrrrz e6562de1f5 Reformat XAML 2026-02-24 21:07:48 +02:00
Tyrrrz 5f8054e2f0 Fix alingment issues when dealing with inlines 2026-02-24 21:05:03 +02:00
Copilot 12d98e9ab0 Add localization (#1482)
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-02-24 20:49:32 +02:00
Copilot 18086aa209 Fix second dialog not appearing when dialogs are shown sequentially (#1481)
* Initial plan

* Add Task.Yield() to DialogManager to fix sequential dialog display

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-23 07:32:56 +02:00
Copilot dd7196b6a5 Resolve thread mentions on demand (#1480)
* Initial plan

* Fix unresolved thread mentions in HTML export (#1261)

- Add TryGetChannelAsync to DiscordClient for on-demand channel/thread lookup
- Add PopulateChannelAsync to ExportContext with negative caching
- Update HtmlMarkdownVisitor to resolve thread mentions on demand

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Refactor GetChannelAsync to use TryGetChannelAsync for parent resolution

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Add test for thread mention resolution in HTML export

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

* Apply PopulateChannelAsync to PlainTextMarkdownVisitor; add JSON thread mention test

Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Tyrrrz <1935960+Tyrrrz@users.noreply.github.com>
2026-02-21 23:21:47 +02:00
237 changed files with 3305 additions and 1757 deletions
+2 -2
View File
@@ -17,7 +17,7 @@ Note the `:stable` tag. DiscordChatExporter images are tagged according to the f
- `stable` — latest stable version release. This tag is updated with each release of a new project version. Recommended for personal use.
- `x.y.z` (e.g. `2.30.1`) — specific stable version release. This tag is pushed when the corresponding version is released and never updated thereafter. Recommended for use in automation scenarios.
- `latest` — latest (potentially unstable) build. This tag is updated with each new commit to the `master` branch. Not recommended, unless you want to test a new feature that has not been released in a stable version yet.
- `latest` — latest (potentially unstable) build. This tag is updated with each new commit to the `prime` branch. Not recommended, unless you want to test a new feature that has not been released in a stable version yet.
You can see all available tags [here](https://hub.docker.com/r/tyrrrz/discordchatexporter/tags?ordering=name).
@@ -57,7 +57,7 @@ You can also use the current working directory as the output directory by specif
- `-v $PWD:/out` in Bash
- `-v $pwd.Path:/out` in PowerShell
For more information, please refer to the [Dockerfile](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/DiscordChatExporter.Cli.dockerfile) and [Docker documentation](https://docs.docker.com/engine/reference/run).
For more information, please refer to the [Dockerfile](https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/DiscordChatExporter.Cli.dockerfile) and [Docker documentation](https://docs.docker.com/engine/reference/run).
To get your Token and Channel IDs, please refer to [this page](Token-and-IDs.md).
+1 -1
View File
@@ -11,7 +11,7 @@
```bash
#!/bin/bash
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
TOKEN=tokenhere
CHANNELID=channelhere
+1 -1
View File
@@ -12,7 +12,7 @@
```bash
#!/bin/bash
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
TOKEN=tokenhere
CHANNELID=channelhere
+1 -1
View File
@@ -5,7 +5,7 @@
1. Open a text editor such as Notepad and paste:
```console
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
# Info: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
$TOKEN = "tokenhere"
$CHANNEL = "channelhere"
+2 -2
View File
@@ -27,7 +27,7 @@ Prerequisite step: Navigate to [discord.com](https://discord.com) and login.
3. Type
```js
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.$8&&x.exports.LP&&x.exports.gK){m=x;break}}}]);m&&console.log("Token:",m.exports.LP());
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.getToken){m=x;break}}}]);m&&console.log("Token:",m.exports.getToken());
```
into the console and press <kbd>Enter</kbd>. The console will display your user token.
@@ -120,7 +120,7 @@ Prerequisite step: Navigate to [discord.com](https://discord.com) and login.
1. Type
```js
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.$8&&x.exports.LP&&x.exports.gK){m=x;break}}}]);m&&console.log("Token:",m.exports.LP());
let m;webpackChunkdiscord_app.push([[Math.random()],{},e=>{for(let i in e.c){let x=e.c[i];if(x?.exports?.getToken){m=x;break}}}]);m&&console.log("Token:",m.exports.getToken());
```
into the console and press <kbd>Enter</kbd>. The console will display your user token.
+11 -12
View File
@@ -29,21 +29,20 @@ Type the following command in your terminal of choice, then press ENTER to run i
> **Note**:
> On Windows, if you're using the default Command Prompt (`cmd`), omit the leading `./` at the start of the command.
> **Docker** users, please refer to the [Docker usage instructions](Docker.md).
## CLI commands
| Command | Description |
|-------------------------|------------------------------------------------------|
| export | Exports a channel |
| exportdm | Exports all direct message channels |
| exportguild | Exports all channels within the specified server |
| exportall | Exports all accessible channels |
| channels | Outputs the list of channels in the given server |
| dm | Outputs the list of direct message channels |
| guilds | Outputs the list of accessible servers |
| guide | Explains how to obtain token, server, and channel ID |
| Command | Description |
| ----------- | ---------------------------------------------------- |
| export | Exports a channel |
| exportdm | Exports all direct message channels |
| exportguild | Exports all channels within the specified server |
| exportall | Exports all accessible channels |
| channels | Outputs the list of channels in the given server |
| dm | Outputs the list of direct message channels |
| guilds | Outputs the list of accessible servers |
| guide | Explains how to obtain token, server, and channel ID |
To use the commands, you'll need a token. For the instructions on how to get a token, please refer to [this page](Token-and-IDs.md), or run `./DiscordChatExporter.Cli guide`.
@@ -222,7 +221,7 @@ Use `--filter` to filter what messages are included in the export.
./DiscordChatExporter.Cli export -t "mfa.Ifrn" -c 53555 --filter "from:Tyrrrz has:image"
```
Documentation on message filter syntax can be found [here](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Message-filters.md).
Documentation on message filter syntax can be found [here](https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs/Message-filters.md).
### Export channels from a specific server
+3 -3
View File
@@ -1,10 +1,10 @@
blank_issues_enabled: false
contact_links:
- name: ⚠ Feature request
url: https://github.com/Tyrrrz/.github/blob/master/docs/project-status.md
about: Sorry, but this project is in maintenance mode and no longer accepts new feature requests.
url: https://github.com/Tyrrrz/.github/blob/prime/docs/project-status.md
about: Sorry, but this project is in maintenance mode and currently doesn't accept new feature requests.
- name: 📖 Documentation
url: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs
url: https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs
about: Find usage guides and frequently asked questions.
- name: 🗨 Discussions
url: https://github.com/Tyrrrz/DiscordChatExporter/discussions/new
+18
View File
@@ -0,0 +1,18 @@
changelog:
exclude:
authors:
- dependabot
- dependabot[bot]
categories:
- title: Enhancements
labels:
- enhancement
- title: Bugs
labels:
- bug
- title: Other
labels:
- "*"
+7 -7
View File
@@ -4,12 +4,12 @@ on:
workflow_dispatch:
push:
branches:
- master
- prime
tags:
- "*"
pull_request:
branches:
- master
- prime
jobs:
# Outputs from this job aren't really used, but it's here to verify that the Dockerfile builds correctly
@@ -26,7 +26,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Build image
run: >
@@ -37,15 +37,15 @@ jobs:
--output type=tar,dest=DiscordChatExporter.Cli.Docker.tar
- name: Upload image
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: DiscordChatExporter.Cli.Docker
path: DiscordChatExporter.Cli.Docker.tar
if-no-files-found: error
deploy:
# Deploy to DockerHub only on tag push or master branch push
if: ${{ github.ref_type == 'tag' || github.ref_type == 'branch' && github.ref_name == 'master' }}
# Deploy to DockerHub only on tag push or prime branch push
if: ${{ github.ref_type == 'tag' || github.ref_type == 'branch' && github.ref_name == 'prime' }}
runs-on: ubuntu-latest
timeout-minutes: 10
@@ -58,7 +58,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Login to DockerHub
run: >
+10 -9
View File
@@ -4,12 +4,12 @@ on:
workflow_dispatch:
push:
branches:
- master
- prime
tags:
- "*"
pull_request:
branches:
- master
- prime
env:
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
@@ -29,7 +29,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install .NET
uses: actions/setup-dotnet@baa11fbfe1d6520db94683bd5c7a3818018e4309 # v5.1.0
uses: actions/setup-dotnet@9a946fdbd5fb07b82b2f5a4466058b876ab72bb2 # v5.3.0
# Build the project separately to discern between build and format errors
- name: Build
@@ -65,7 +65,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install .NET
uses: actions/setup-dotnet@baa11fbfe1d6520db94683bd5c7a3818018e4309 # v5.1.0
uses: actions/setup-dotnet@9a946fdbd5fb07b82b2f5a4466058b876ab72bb2 # v5.3.0
- name: Run tests
env:
@@ -81,7 +81,7 @@ jobs:
DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
- name: Upload coverage
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
with:
token: ${{ secrets.CODECOV_TOKEN }}
@@ -120,13 +120,14 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install .NET
uses: actions/setup-dotnet@baa11fbfe1d6520db94683bd5c7a3818018e4309 # v5.1.0
uses: actions/setup-dotnet@9a946fdbd5fb07b82b2f5a4466058b876ab72bb2 # v5.3.0
- name: Publish app
run: >
dotnet publish ${{ matrix.app }}
-p:Version=${{ github.ref_type == 'tag' && github.ref_name || format('999.9.9-ci-{0}', github.sha) }}
-p:CSharpier_Bypass=true
-p:EncryptionSalt=${{ secrets.ENCRYPTION_SALT || 'HimalayanPinkSalt' }}
-p:PublishMacOSBundle=${{ startsWith(matrix.rid, 'osx-') }}
--output ${{ matrix.app }}/bin/publish/
--configuration Release
@@ -134,7 +135,7 @@ jobs:
--self-contained
- name: Upload app binaries
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.asset }}.${{ matrix.rid }}
path: ${{ matrix.app }}/bin/publish/
@@ -199,7 +200,7 @@ jobs:
steps:
- name: Download app binaries
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ${{ matrix.asset }}.${{ matrix.rid }}
path: ${{ matrix.app }}/
@@ -235,7 +236,7 @@ jobs:
steps:
- name: Notify Discord
uses: tyrrrz/action-http-request@1dd7ad841a34b9299f3741f7c7399f9feefdfb08 # 1.1.3
uses: tyrrrz/action-http-request@110f1a0f0f7e91c3b2de349539249f4573c243fa # 1.1.5
with:
url: ${{ secrets.DISCORD_WEBHOOK }}
method: POST
+2 -4
View File
@@ -1,14 +1,12 @@
<Project>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Version>999.9.9-dev</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Oleksii Holub</Copyright>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<ILLinkTreatWarningsAsErrors>false</ILLinkTreatWarningsAsErrors>
</PropertyGroup>
</Project>
</Project>
+46
View File
@@ -0,0 +1,46 @@
<Project>
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="AngleSharp" Version="1.4.0" />
<PackageVersion Include="AsyncImageLoader.Avalonia" Version="3.8.0" />
<PackageVersion Include="AsyncKeyedLock" Version="8.0.2" />
<PackageVersion Include="Avalonia" Version="12.0.4" />
<PackageVersion Include="Avalonia.Desktop" Version="12.0.4" />
<PackageVersion Include="CliFx" Version="3.0.0" />
<PackageVersion Include="Cogwheel" Version="2.1.1" />
<PackageVersion Include="CommunityToolkit.Mvvm" Version="8.4.2" />
<PackageVersion Include="coverlet.collector" Version="10.0.1" />
<PackageVersion Include="CSharpier.MsBuild" Version="1.2.6" />
<PackageVersion Include="Deorcify" Version="1.1.0" />
<PackageVersion Include="DialogHost.Avalonia" Version="0.12.2" />
<PackageVersion Include="FluentAssertions" Version="8.10.0" />
<PackageVersion Include="GitHubActionsTestLogger" Version="3.0.4" />
<PackageVersion Include="Gress" Version="2.2.0" />
<PackageVersion Include="JsonExtensions" Version="1.2.0" />
<PackageVersion Include="Markdig" Version="1.2.0" />
<PackageVersion Include="Material.Avalonia" Version="3.16.1" />
<PackageVersion Include="Material.Icons.Avalonia" Version="3.0.2" />
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="10.0.8" />
<PackageVersion
Include="Microsoft.Extensions.Configuration.EnvironmentVariables"
Version="10.0.8"
/>
<PackageVersion Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.8" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.8" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.6.0" />
<PackageVersion Include="Onova" Version="2.6.13" />
<PackageVersion Include="Polly" Version="8.6.6" />
<PackageVersion Include="PowerKit" Version="1.2.0" />
<PackageVersion Include="RazorBlade" Version="1.0.0" />
<PackageVersion Include="Spectre.Console" Version="0.55.2" />
<PackageVersion Include="Superpower" Version="3.1.0" />
<PackageVersion Include="ThisAssembly.Project" Version="2.1.2" />
<PackageVersion Include="WebMarkupMin.Core" Version="2.21.0" />
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
<PackageVersion Include="YoutubeExplode" Version="6.6.0" />
</ItemGroup>
</Project>
@@ -10,24 +10,22 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="AngleSharp" Version="1.4.0" />
<PackageReference Include="coverlet.collector" Version="6.0.4" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" Version="1.2.5" PrivateAssets="all" />
<PackageReference Include="FluentAssertions" Version="8.8.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="3.0.1" PrivateAssets="all" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="10.0.2" />
<PackageReference
Include="Microsoft.Extensions.Configuration.EnvironmentVariables"
Version="10.0.2"
/>
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.2" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5" PrivateAssets="all" />
<PackageReference Include="AngleSharp" />
<PackageReference Include="coverlet.collector" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="FluentAssertions" />
<PackageReference Include="GitHubActionsTestLogger" PrivateAssets="all" />
<PackageReference Include="JsonExtensions" />
<PackageReference Include="Microsoft.Extensions.Configuration" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" />
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="PowerKit" PrivateAssets="all" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Cli\DiscordChatExporter.Cli.csproj" />
<ProjectReference Include="../DiscordChatExporter.Cli/DiscordChatExporter.Cli.csproj" />
</ItemGroup>
</Project>
@@ -1,4 +1,4 @@
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Cli.Tests.Infra;
@@ -16,6 +16,8 @@ public static class ChannelIds
public static Snowflake FilterTestCases { get; } = Snowflake.Parse("866744075033641020");
public static Snowflake ForwardTestCases { get; } = Snowflake.Parse("1455202357204877477");
public static Snowflake MarkdownTestCases { get; } = Snowflake.Parse("866459526819348521");
public static Snowflake MentionTestCases { get; } = Snowflake.Parse("866458801389174794");
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Reflection;
using Microsoft.Extensions.Configuration;
+1 -1
View File
@@ -1,4 +1,4 @@
# DiscordChatExporter Tests
# DiscordChatExporter Tests
This test suite runs against a real Discord server, specifically created to exercise different behaviors required by the test scenarios.
In order to run these tests locally, you need to join the test server and configure your authentication token.
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using Xunit;
@@ -1,15 +1,15 @@
using System;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,15 +1,15 @@
using System;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Filtering;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
@@ -1,8 +1,14 @@
using System.Linq;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -44,4 +50,41 @@ public class HtmlContentSpecs
"Yeet"
);
}
[Fact]
public async Task I_can_export_a_channel_in_the_HTML_format_in_the_reverse_order()
{
// Arrange
using var file = TempFile.Create();
// Act
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.HtmlDark,
OutputPath = file.Path,
Locale = "en-US",
IsUtcNormalizationEnabled = true,
IsReverseMessageOrder = true,
}.ExecuteAsync(new FakeConsole());
var document = Html.Parse(await File.ReadAllTextAsync(file.Path));
var messages = document.QuerySelectorAll("[data-message-id]").ToArray();
// Assert
messages
.Select(e => e.GetAttribute("data-message-id"))
.Should()
.Equal(
"885169254029213696",
"868505973294268457",
"868505969821364245",
"868505966528835604",
"868490009366396958",
"866732113319428096",
"866710679758045195",
"866674314627121232"
);
}
}
@@ -1,11 +1,11 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils.Extensions;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -0,0 +1,29 @@
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlForwardSpecs
{
[Fact]
public async Task I_can_export_a_channel_that_contains_a_forwarded_message()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.ForwardTestCases,
Snowflake.Parse("1455202427115536514")
);
// Assert
message
.Text()
.ReplaceWhiteSpace(' ')
.Should()
.ContainAll("Forwarded", @"¯\_(ツ)_/¯", "12/29/2025 2:14 PM");
}
}
@@ -1,4 +1,4 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
@@ -8,6 +8,7 @@ using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -1,9 +1,9 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils.Extensions;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -22,11 +22,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Default timestamp: 2/12/2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -39,8 +42,11 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace().Should().Contain("Short time timestamp: 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Short time timestamp: 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -53,8 +59,11 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace().Should().Contain("Long time timestamp: 1:36:12 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Long time timestamp: 1:36:12 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -67,8 +76,11 @@ public class HtmlMarkdownSpecs
);
// Assert
message.Text().ReplaceWhiteSpace().Should().Contain("Short date timestamp: 2/12/2023");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message.Text().ReplaceWhiteSpace(' ').Should().Contain("Short date timestamp: 2/12/2023");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -83,11 +95,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Long date timestamp: Sunday, February 12, 2023");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -102,11 +117,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Full timestamp: Sunday, February 12, 2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -121,11 +139,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Full long timestamp: Sunday, February 12, 2023 1:36:12 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -140,11 +161,14 @@ public class HtmlMarkdownSpecs
// Assert
message
.Text()
.ReplaceWhiteSpace()
.ReplaceWhiteSpace(' ')
.Should()
.Contain("Relative timestamp: 2/12/2023 1:36 PM");
message.InnerHtml.ReplaceWhiteSpace().Should().Contain("Sunday, February 12, 2023 1:36 PM");
message
.InnerHtml.ReplaceWhiteSpace(' ')
.Should()
.Contain("Sunday, February 12, 2023 1:36 PM");
}
[Fact]
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -61,4 +61,17 @@ public class HtmlMentionSpecs
// Assert
message.Text().Should().Contain("Role mention: @Role 1");
}
[Fact]
public async Task I_can_export_a_channel_that_contains_a_message_with_a_thread_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.MentionTestCases,
Snowflake.Parse("1474874276828938290")
);
// Assert
message.Text().Should().Contain("Thread mention: #Thread starting message");
}
}
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,7 +1,13 @@
using System.Linq;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -43,4 +49,43 @@ public class JsonContentSpecs
"Yeet"
);
}
[Fact]
public async Task I_can_export_a_channel_in_the_JSON_format_in_the_reverse_order()
{
// Arrange
using var file = TempFile.Create();
// Act
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
Locale = "en-US",
IsUtcNormalizationEnabled = true,
IsReverseMessageOrder = true,
}.ExecuteAsync(new FakeConsole());
var messages = Json.Parse(await File.ReadAllTextAsync(file.Path))
.GetProperty("messages")
.EnumerateArray()
.ToArray();
// Assert
messages
.Select(j => j.GetProperty("id").GetString())
.Should()
.Equal(
"885169254029213696",
"868505973294268457",
"868505969821364245",
"868505966528835604",
"868490009366396958",
"866732113319428096",
"866710679758045195",
"866674314627121232"
);
}
}
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -0,0 +1,33 @@
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
public class JsonForwardSpecs
{
[Fact]
public async Task I_can_export_a_channel_that_contains_a_forwarded_message()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
ChannelIds.ForwardTestCases,
Snowflake.Parse("1455202427115536514")
);
// Assert
var reference = message.GetProperty("reference");
reference.GetProperty("type").GetString().Should().Be("Forward");
reference.GetProperty("guildId").GetString().Should().Be("869237470565392384");
var forwardedMessage = message.GetProperty("forwardedMessage");
forwardedMessage.GetProperty("content").GetString().Should().Contain(@"¯\_(ツ)_/¯");
forwardedMessage
.GetProperty("timestamp")
.GetString()
.Should()
.StartWith("2025-12-28T22:52:42.175+00:00");
}
}
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -75,4 +75,21 @@ public class JsonMentionSpecs
// Assert
message.GetProperty("content").GetString().Should().Be("Role mention: @Role 1");
}
[Fact]
public async Task I_can_export_a_channel_that_contains_a_message_with_a_thread_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
ChannelIds.MentionTestCases,
Snowflake.Parse("1474874276828938290")
);
// Assert
message
.GetProperty("content")
.GetString()
.Should()
.Be("Thread mention: #Thread starting message");
}
}
@@ -1,4 +1,4 @@
using System.Linq;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
@@ -1,12 +1,12 @@
using System.IO;
using System.IO;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Partitioning;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -17,7 +17,7 @@ public class PartitioningSpecs
public async Task I_can_export_a_channel_with_partitioning_based_on_message_count()
{
// Arrange
using var dir = TempDir.Create();
using var dir = TempDirectory.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -38,7 +38,7 @@ public class PartitioningSpecs
public async Task I_can_export_a_channel_with_partitioning_based_on_file_size()
{
// Arrange
using var dir = TempDir.Create();
using var dir = TempDirectory.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -1,4 +1,4 @@
using System.Threading.Tasks;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using Xunit;
@@ -0,0 +1,23 @@
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Tests.Infra;
using FluentAssertions;
using PowerKit.Extensions;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
public class PlainTextForwardSpecs
{
[Fact]
public async Task I_can_export_a_channel_that_contains_a_forwarded_message()
{
// Act
var document = await ExportWrapper.ExportAsPlainTextAsync(ChannelIds.ForwardTestCases);
// Assert
document
.ReplaceWhiteSpace(' ')
.Should()
.ContainAll("{Forwarded Message}", @"¯\_(ツ)_/¯", "12/28/2025 10:52 PM");
}
}
@@ -1,4 +1,4 @@
using System.IO;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
@@ -7,6 +7,7 @@ using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using PowerKit;
using Xunit;
namespace DiscordChatExporter.Cli.Tests.Specs;
@@ -17,7 +18,7 @@ public class SelfContainedSpecs
public async Task I_can_export_a_channel_and_download_all_referenced_assets()
{
// Arrange
using var dir = TempDir.Create();
using var dir = TempDirectory.Create();
var filePath = Path.Combine(dir.Path, "output.html");
// Act
@@ -1,19 +0,0 @@
using System.Text;
namespace DiscordChatExporter.Cli.Tests.Utils.Extensions;
internal static class StringExtensions
{
extension(string str)
{
public string ReplaceWhiteSpace(string replacement = " ")
{
var buffer = new StringBuilder(str.Length);
foreach (var ch in str)
buffer.Append(char.IsWhiteSpace(ch) ? replacement : ch);
return buffer.ToString();
}
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
using AngleSharp.Html.Dom;
using AngleSharp.Html.Dom;
using AngleSharp.Html.Parser;
namespace DiscordChatExporter.Cli.Tests.Utils;
@@ -1,36 +0,0 @@
using System;
using System.IO;
using System.Reflection;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempDir(string path) : IDisposable
{
public string Path { get; } = path;
public void Dispose()
{
try
{
Directory.Delete(Path, true);
}
catch (DirectoryNotFoundException) { }
}
}
internal partial class TempDir
{
public static TempDir Create()
{
var dirPath = System.IO.Path.Combine(
System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
?? Directory.GetCurrentDirectory(),
"Temp",
Guid.NewGuid().ToString()
);
Directory.CreateDirectory(dirPath);
return new TempDir(dirPath);
}
}
@@ -1,37 +0,0 @@
using System;
using System.IO;
using System.Reflection;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempFile(string path) : IDisposable
{
public string Path { get; } = path;
public void Dispose()
{
try
{
File.Delete(Path);
}
catch (FileNotFoundException) { }
}
}
internal partial class TempFile
{
public static TempFile Create()
{
var dirPath = System.IO.Path.Combine(
System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
?? Directory.GetCurrentDirectory(),
"Temp"
);
Directory.CreateDirectory(dirPath);
var filePath = System.IO.Path.Combine(dirPath, Guid.NewGuid() + ".tmp");
return new TempFile(filePath);
}
}
@@ -1,4 +1,4 @@
{
{
"$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
"methodDisplayOptions": "all",
"methodDisplay": "method"
+1
View File
@@ -15,6 +15,7 @@ WORKDIR /tmp/app
COPY favicon.ico .
COPY NuGet.config .
COPY Directory.Build.props .
COPY Directory.Packages.props .
COPY DiscordChatExporter.Core DiscordChatExporter.Core
COPY DiscordChatExporter.Cli DiscordChatExporter.Cli
@@ -1,8 +1,8 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Threading.Tasks;
using CliFx;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils;
@@ -17,23 +17,22 @@ public abstract class DiscordCommandBase : ICommand
EnvironmentVariable = "DISCORD_TOKEN",
Description = "Authentication token."
)]
public required string Token { get; init; }
public required string Token { get; set; }
[Obsolete("This option doesn't do anything. Kept for backwards compatibility.")]
[CommandOption(
"bot",
'b',
EnvironmentVariable = "DISCORD_TOKEN_BOT",
Description = "This option doesn't do anything. Kept for backwards compatibility."
)]
public bool IsBotToken { get; init; } = false;
public bool IsBotToken { get; set; } = false;
[CommandOption(
"respect-rate-limits",
Description = "Whether to respect advisory rate limits. "
+ "If disabled, only hard rate limits (i.e. 429 responses) will be respected."
)]
public bool ShouldRespectRateLimits { get; init; } = true;
public bool ShouldRespectRateLimits { get; set; } = true;
[field: AllowNull, MaybeNull]
protected DiscordClient Discord =>
@@ -1,12 +1,12 @@
using System;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Exceptions;
using CliFx;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Cli.Commands.Shared;
@@ -37,23 +37,23 @@ public abstract class ExportCommandBase : DiscordCommandBase
get;
// Handle ~/ in paths on Unix systems
// https://github.com/Tyrrrz/DiscordChatExporter/pull/903
init => field = Path.GetFullPath(value);
set => field = Path.GetFullPath(value);
} = Directory.GetCurrentDirectory();
[CommandOption("format", 'f', Description = "Export format.")]
public ExportFormat ExportFormat { get; init; } = ExportFormat.HtmlDark;
public ExportFormat ExportFormat { get; set; } = ExportFormat.HtmlDark;
[CommandOption(
"after",
Description = "Only include messages sent after this date or message ID."
)]
public Snowflake? After { get; init; }
public Snowflake? After { get; set; }
[CommandOption(
"before",
Description = "Only include messages sent before this date or message ID."
)]
public Snowflake? Before { get; init; }
public Snowflake? Before { get; set; }
[CommandOption(
"partition",
@@ -61,45 +61,51 @@ public abstract class ExportCommandBase : DiscordCommandBase
Description = "Split the output into partitions, each limited to the specified "
+ "number of messages (e.g. '100') or file size (e.g. '10mb')."
)]
public PartitionLimit PartitionLimit { get; init; } = PartitionLimit.Null;
public PartitionLimit PartitionLimit { get; set; } = PartitionLimit.Null;
[CommandOption(
"include-threads",
Description = "Which types of threads should be included.",
Converter = typeof(ThreadInclusionModeBindingConverter)
Converter = typeof(ThreadInclusionModeInputConverter)
)]
public ThreadInclusionMode ThreadInclusionMode { get; init; } = ThreadInclusionMode.None;
public ThreadInclusionMode ThreadInclusionMode { get; set; } = ThreadInclusionMode.None;
[CommandOption(
"filter",
Description = "Only include messages that satisfy this filter. "
+ "See the documentation for more info."
)]
public MessageFilter MessageFilter { get; init; } = MessageFilter.Null;
public MessageFilter MessageFilter { get; set; } = MessageFilter.Null;
[CommandOption(
"parallel",
Description = "Limits how many channels can be exported in parallel."
)]
public int ParallelLimit { get; init; } = 1;
public int ParallelLimit { get; set; } = 1;
[CommandOption(
"reverse",
Description = "Export messages in reverse chronological order (newest first)."
)]
public bool IsReverseMessageOrder { get; set; }
[CommandOption(
"markdown",
Description = "Process markdown, mentions, and other special tokens."
)]
public bool ShouldFormatMarkdown { get; init; } = true;
public bool ShouldFormatMarkdown { get; set; } = true;
[CommandOption(
"media",
Description = "Download assets referenced by the export (user avatars, attached files, embedded images, etc.)."
)]
public bool ShouldDownloadAssets { get; init; }
public bool ShouldDownloadAssets { get; set; }
[CommandOption(
"reuse-media",
Description = "Reuse previously downloaded assets to avoid redundant requests."
)]
public bool ShouldReuseAssets { get; init; } = false;
public bool ShouldReuseAssets { get; set; } = false;
[CommandOption(
"media-dir",
@@ -111,34 +117,33 @@ public abstract class ExportCommandBase : DiscordCommandBase
get;
// Handle ~/ in paths on Unix systems
// https://github.com/Tyrrrz/DiscordChatExporter/pull/903
init => field = value is not null ? Path.GetFullPath(value) : null;
set => field = value is not null ? Path.GetFullPath(value) : null;
}
[Obsolete("This option doesn't do anything. Kept for backwards compatibility.")]
[CommandOption(
"dateformat",
Description = "This option doesn't do anything. Kept for backwards compatibility."
)]
public string DateFormat { get; init; } = "MM/dd/yyyy h:mm tt";
public string DateFormat { get; set; } = "MM/dd/yyyy h:mm tt";
[CommandOption(
"locale",
Description = "Locale to use when formatting dates and numbers. "
+ "If not specified, the default system locale will be used."
)]
public string? Locale { get; init; }
public string? Locale { get; set; }
[CommandOption("utc", Description = "Normalize all timestamps to UTC+0.")]
public bool IsUtcNormalizationEnabled { get; init; } = false;
public bool IsUtcNormalizationEnabled { get; set; } = false;
[CommandOption(
"fuck-russia",
EnvironmentVariable = "FUCK_RUSSIA",
Description = "Don't print the Support Ukraine message to the console.",
// Use a converter to accept '1' as 'true' to reuse the existing environment variable
Converter = typeof(TruthyBooleanBindingConverter)
Converter = typeof(TruthyBooleanInputConverter)
)]
public bool IsUkraineSupportMessageDisabled { get; init; } = false;
public bool IsUkraineSupportMessageDisabled { get; set; } = false;
[field: AllowNull, MaybeNull]
protected ChannelExporter Exporter => field ??= new ChannelExporter(Discord);
@@ -267,6 +272,7 @@ public abstract class ExportCommandBase : DiscordCommandBase
Before,
PartitionLimit,
MessageFilter,
IsReverseMessageOrder,
ShouldFormatMarkdown,
ShouldDownloadAssets,
ShouldReuseAssets,
@@ -1,10 +1,10 @@
using System;
using CliFx.Extensibility;
using System;
using CliFx.Activation;
using DiscordChatExporter.Cli.Commands.Shared;
namespace DiscordChatExporter.Cli.Commands.Converters;
internal class ThreadInclusionModeBindingConverter : BindingConverter<ThreadInclusionMode>
internal class ThreadInclusionModeInputConverter : ScalarInputConverter<ThreadInclusionMode>
{
public override ThreadInclusionMode Convert(string? rawValue)
{
@@ -1,9 +1,9 @@
using System.Globalization;
using CliFx.Extensibility;
using System.Globalization;
using CliFx.Activation;
namespace DiscordChatExporter.Cli.Commands.Converters;
internal class TruthyBooleanBindingConverter : BindingConverter<bool>
internal class TruthyBooleanInputConverter : ScalarInputConverter<bool>
{
public override bool Convert(string? rawValue)
{
@@ -2,7 +2,7 @@ using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
@@ -14,23 +14,23 @@ using Spectre.Console;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportall", Description = "Exports all accessible channels.")]
public class ExportAllCommand : ExportCommandBase
public partial class ExportAllCommand : ExportCommandBase
{
[CommandOption("include-dm", Description = "Include direct message channels.")]
public bool IncludeDirectChannels { get; init; } = true;
public bool IncludeDirectChannels { get; set; } = true;
[CommandOption("include-guilds", Description = "Include server channels.")]
public bool IncludeGuildChannels { get; init; } = true;
public bool IncludeGuildChannels { get; set; } = true;
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; init; } = true;
public bool IncludeVoiceChannels { get; set; } = true;
[CommandOption(
"data-package",
Description = "Path to the personal data package (ZIP file) requested from Discord. "
+ "If provided, only channels referenced in the dump will be exported."
)]
public string? DataPackageFilePath { get; init; }
public string? DataPackageFilePath { get; set; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,16 +1,16 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("export", Description = "Exports one or multiple channels.")]
public class ExportChannelsCommand : ExportCommandBase
public partial class ExportChannelsCommand : ExportCommandBase
{
// TODO: change this to plural (breaking change)
[CommandOption(
@@ -19,7 +19,7 @@ public class ExportChannelsCommand : ExportCommandBase
Description = "Channel ID(s). "
+ "If provided with category ID(s), all channels inside those categories will be exported."
)]
public required IReadOnlyList<Snowflake> ChannelIds { get; init; }
public required IReadOnlyList<Snowflake> ChannelIds { get; set; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,14 +1,14 @@
using System.Threading.Tasks;
using CliFx.Attributes;
using System.Threading.Tasks;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportdm", Description = "Exports all direct message channels.")]
public class ExportDirectMessagesCommand : ExportCommandBase
public partial class ExportDirectMessagesCommand : ExportCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,6 +1,6 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Utils.Extensions;
@@ -11,13 +11,13 @@ using Spectre.Console;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportguild", Description = "Exports all channels within the specified server.")]
public class ExportGuildCommand : ExportCommandBase
public partial class ExportGuildCommand : ExportCommandBase
{
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; init; }
public required Snowflake GuildId { get; set; }
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; init; } = true;
public bool IncludeVoiceChannels { get; set; } = true;
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,31 +1,31 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Cli.Commands.Shared;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("channels", Description = "Get the list of channels in a server.")]
public class GetChannelsCommand : DiscordCommandBase
public partial class GetChannelsCommand : DiscordCommandBase
{
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; init; }
public required Snowflake GuildId { get; set; }
[CommandOption("include-vc", Description = "Include voice channels.")]
public bool IncludeVoiceChannels { get; init; } = true;
public bool IncludeVoiceChannels { get; set; } = true;
[CommandOption(
"include-threads",
Description = "Which types of threads should be included.",
Converter = typeof(ThreadInclusionModeBindingConverter)
Converter = typeof(ThreadInclusionModeInputConverter)
)]
public ThreadInclusionMode ThreadInclusionMode { get; init; } = ThreadInclusionMode.None;
public ThreadInclusionMode ThreadInclusionMode { get; set; } = ThreadInclusionMode.None;
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,16 +1,16 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("dm", Description = "Gets the list of all direct message channels.")]
public class GetDirectChannelsCommand : DiscordCommandBase
public partial class GetDirectChannelsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,16 +1,16 @@
using System;
using System;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Utils.Extensions;
using PowerKit.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("guilds", Description = "Gets the list of accessible servers.")]
public class GetGuildsCommand : DiscordCommandBase
public partial class GetGuildsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -1,13 +1,13 @@
using System;
using System.Threading.Tasks;
using CliFx;
using CliFx.Attributes;
using CliFx.Binding;
using CliFx.Infrastructure;
namespace DiscordChatExporter.Cli.Commands;
[Command("guide", Description = "Explains how to obtain the token, server or channel ID.")]
public class GuideCommand : ICommand
public partial class GuideCommand : ICommand
{
public ValueTask ExecuteAsync(IConsole console)
{
@@ -74,7 +74,7 @@ public class GuideCommand : ICommand
using (console.WithForegroundColor(ConsoleColor.DarkCyan))
{
console.Output.WriteLine(
"https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs"
"https://github.com/Tyrrrz/DiscordChatExporter/blob/prime/.docs"
);
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Cli.Commands.Shared;
namespace DiscordChatExporter.Cli.Commands.Shared;
public enum ThreadInclusionMode
{
@@ -1,26 +1,21 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<ApplicationIcon>..\favicon.ico</ApplicationIcon>
<ApplicationIcon>../favicon.ico</ApplicationIcon>
<PublishTrimmed>true</PublishTrimmed>
<CopyOutputSymbolsToPublishDirectory>false</CopyOutputSymbolsToPublishDirectory>
</PropertyGroup>
<!-- HACK: Disable trim warnings because they seem to break when the code contains C# 14 extension blocks -->
<PropertyGroup>
<EnableTrimAnalyzer>false</EnableTrimAnalyzer>
<EnableAotAnalyzer>false</EnableAotAnalyzer>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliFx" Version="2.3.6" />
<PackageReference Include="CSharpier.MsBuild" Version="1.2.5" PrivateAssets="all" />
<PackageReference Include="Deorcify" Version="1.1.0" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="Spectre.Console" Version="0.54.0" />
<PackageReference Include="CliFx" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="Deorcify" PrivateAssets="all" />
<PackageReference Include="Gress" />
<PackageReference Include="PowerKit" PrivateAssets="all" />
<PackageReference Include="Spectre.Console" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core\DiscordChatExporter.Core.csproj" />
<ProjectReference Include="../DiscordChatExporter.Core/DiscordChatExporter.Core.csproj" />
</ItemGroup>
</Project>
+2 -30
View File
@@ -1,41 +1,13 @@
using System.Diagnostics.CodeAnalysis;
using System.Threading.Tasks;
using CliFx;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Core.Exporting.Filtering;
using DiscordChatExporter.Core.Exporting.Partitioning;
namespace DiscordChatExporter.Cli;
public static class Program
{
// Explicit references because CliFx relies on reflection and we're publishing with trimming enabled
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportAllCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportChannelsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportDirectMessagesCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(ExportGuildCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GetChannelsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GetDirectChannelsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GetGuildsCommand))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(GuideCommand))]
[DynamicDependency(
DynamicallyAccessedMemberTypes.All,
typeof(ThreadInclusionModeBindingConverter)
)]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(TruthyBooleanBindingConverter))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(PartitionLimit))]
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(MessageFilter))]
public static async Task<int> Main(string[] args) =>
await new CliApplicationBuilder()
.AddCommand<ExportAllCommand>()
.AddCommand<ExportChannelsCommand>()
.AddCommand<ExportDirectMessagesCommand>()
.AddCommand<ExportGuildCommand>()
.AddCommand<GetChannelsCommand>()
.AddCommand<GetDirectChannelsCommand>()
.AddCommand<GetGuildsCommand>()
.AddCommand<GuideCommand>()
await new CommandLineApplicationBuilder()
.AddCommandsFromThisAssembly()
.Build()
.RunAsync(args);
}
@@ -1,4 +1,4 @@
using System;
using System;
using System.Threading.Tasks;
using CliFx.Infrastructure;
using Spectre.Console;
@@ -1,6 +1,6 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using System.Text.Json;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System;
using System;
using System.IO;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
public enum ChannelKind
@@ -1,4 +1,4 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data.Common;
namespace DiscordChatExporter.Core.Discord.Data.Common;
public interface IHasId
{
@@ -1,4 +1,4 @@
using System;
using System;
using System.Globalization;
using System.Linq;
@@ -3,8 +3,8 @@ using System.Collections.Generic;
using System.Drawing;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -47,9 +47,7 @@ public partial record Embed
var kind =
json.GetPropertyOrNull("type")
?.GetStringOrNull()
?.Pipe(s =>
Enum.TryParse<EmbedKind>(s, true, out var result) ? result : (EmbedKind?)null
)
.Pipe(s => Enum.ParseOrNull<EmbedKind>(s, true))
?? EmbedKind.Rich;
var url = json.GetPropertyOrNull("url")?.GetNonWhiteSpaceStringOrNull();
@@ -58,7 +56,7 @@ public partial record Embed
var color = json.GetPropertyOrNull("color")
?.GetInt32OrNull()
?.Pipe(System.Drawing.Color.FromArgb)
.ResetAlpha();
.WithFullAlpha();
var author = json.GetPropertyOrNull("author")?.Pipe(EmbedAuthor.Parse);
var description = json.GetPropertyOrNull("description")?.GetStringOrNull();
@@ -1,4 +1,4 @@
using System.Text.RegularExpressions;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -1,4 +1,4 @@
using System.Text.RegularExpressions;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
public partial record YouTubeVideoEmbedProjection(string VideoId)
{
@@ -1,8 +1,8 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
@@ -8855,6 +8855,8 @@ internal static class EmojiIndex
["united_nations"] = "🇺🇳",
};
public static IReadOnlyCollection<string> GetAllNames() => _toCodes.Keys;
public static string? TryGetCode(string name) => _toCodes.GetValueOrDefault(name);
public static string? TryGetName(string code) => _fromCodes.GetValueOrDefault(code);
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,6 +1,6 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using System.Text.Json;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,9 +1,9 @@
using System.Collections.Generic;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,11 +1,11 @@
using System;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -27,9 +27,16 @@ public partial record Message(
IReadOnlyList<User> MentionedUsers,
MessageReference? Reference,
Message? ReferencedMessage,
MessageSnapshot? ForwardedMessage,
Interaction? Interaction
) : IHasId
{
public bool IsEmpty { get; } =
string.IsNullOrWhiteSpace(Content)
&& !Attachments.Any()
&& !Embeds.Any()
&& !Stickers.Any();
public bool IsSystemNotification { get; } =
Kind is >= MessageKind.RecipientAdd and <= MessageKind.ThreadCreated;
@@ -38,11 +45,7 @@ public partial record Message(
// App interactions are rendered as replies in the Discord client, but they are not actually replies
public bool IsReplyLike => IsReply || Interaction is not null;
public bool IsEmpty { get; } =
string.IsNullOrWhiteSpace(Content)
&& !Attachments.Any()
&& !Embeds.Any()
&& !Stickers.Any();
public bool IsForwarded { get; } = Reference?.Kind == MessageReferenceKind.Forward;
public IEnumerable<User> GetReferencedUsers()
{
@@ -171,7 +174,17 @@ public partial record Message
var messageReference = json.GetPropertyOrNull("message_reference")
?.Pipe(MessageReference.Parse);
var referencedMessage = json.GetPropertyOrNull("referenced_message")?.Pipe(Parse);
// Currently Discord only supports 1 snapshot per forward
var forwardedMessage = json.GetPropertyOrNull("message_snapshots")
?.EnumerateArrayOrNull()
?.Select(j => j.GetPropertyOrNull("message"))
.WhereNotNull()
.Select(MessageSnapshot.Parse)
.FirstOrDefault();
var interaction = json.GetPropertyOrNull("interaction")?.Pipe(Interaction.Parse);
return new Message(
@@ -191,6 +204,7 @@ public partial record Message
mentionedUsers,
messageReference,
referencedMessage,
forwardedMessage,
interaction
);
}
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#message-object-message-types
public enum MessageKind
@@ -1,14 +1,23 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#message-object-message-reference-structure
public record MessageReference(Snowflake? MessageId, Snowflake? ChannelId, Snowflake? GuildId)
public record MessageReference(
MessageReferenceKind Kind,
Snowflake? MessageId,
Snowflake? ChannelId,
Snowflake? GuildId
)
{
public static MessageReference Parse(JsonElement json)
{
var kind =
json.GetPropertyOrNull("type")?.GetInt32OrNull()?.Pipe(t => (MessageReferenceKind)t)
?? MessageReferenceKind.Default;
var messageId = json.GetPropertyOrNull("message_id")
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
@@ -21,6 +30,6 @@ public record MessageReference(Snowflake? MessageId, Snowflake? ChannelId, Snowf
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
return new MessageReference(messageId, channelId, guildId);
return new MessageReference(kind, messageId, channelId, guildId);
}
}
@@ -0,0 +1,8 @@
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#message-reference-types
public enum MessageReferenceKind
{
Default = 0,
Forward = 1,
}
@@ -0,0 +1,57 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data;
// https://docs.discord.com/developers/resources/message#message-snapshot-object
public record MessageSnapshot(
DateTimeOffset Timestamp,
DateTimeOffset? EditedTimestamp,
string Content,
IReadOnlyList<Attachment> Attachments,
IReadOnlyList<Embed> Embeds,
IReadOnlyList<Sticker> Stickers
)
{
public static MessageSnapshot Parse(JsonElement json)
{
var timestamp =
json.GetPropertyOrNull("timestamp")?.GetDateTimeOffsetOrNull()
?? DateTimeOffset.MinValue;
var editedTimestamp = json.GetPropertyOrNull("edited_timestamp")?.GetDateTimeOffsetOrNull();
var content = json.GetPropertyOrNull("content")?.GetStringOrNull() ?? "";
var attachments =
json.GetPropertyOrNull("attachments")
?.EnumerateArrayOrNull()
?.Select(Attachment.Parse)
.ToArray()
?? [];
var embeds =
json.GetPropertyOrNull("embeds")?.EnumerateArrayOrNull()?.Select(Embed.Parse).ToArray()
?? [];
var stickers =
json.GetPropertyOrNull("sticker_items")
?.EnumerateArrayOrNull()
?.Select(Sticker.Parse)
.ToArray()
?? [];
return new MessageSnapshot(
timestamp,
editedTimestamp,
content,
attachments,
embeds,
stickers
);
}
}
@@ -1,5 +1,5 @@
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using System.Text.Json;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,8 +1,8 @@
using System.Drawing;
using System.Drawing;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -18,7 +18,7 @@ public record Role(Snowflake Id, string Name, int Position, Color? Color) : IHas
var color = json.GetPropertyOrNull("color")
?.GetInt32OrNull()
?.Pipe(System.Drawing.Color.FromArgb)
.ResetAlpha()
.WithFullAlpha()
.NullIf(c => c.ToRgb() <= 0);
return new Role(id, name, position, color);
@@ -1,8 +1,8 @@
using System;
using System;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Discord.Data;
public enum StickerFormat
{
@@ -1,7 +1,7 @@
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord.Data;
+159 -34
View File
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
@@ -11,10 +11,10 @@ using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using Gress;
using JsonExtensions.Http;
using JsonExtensions.Reading;
using PowerKit.Extensions;
namespace DiscordChatExporter.Core.Discord;
@@ -59,12 +59,12 @@ public class DiscordClient(
{
var remainingRequestCount = response
.Headers.TryGetValue("X-RateLimit-Remaining")
?.Pipe(s => int.Parse(s, CultureInfo.InvariantCulture));
?.Pipe(s => int.ParseOrNull(s, CultureInfo.InvariantCulture));
var resetAfterDelay = response
.Headers.TryGetValue("X-RateLimit-Reset-After")
?.Pipe(s => double.Parse(s, CultureInfo.InvariantCulture))
.Pipe(TimeSpan.FromSeconds);
?.Pipe(s => double.ParseOrNull(s, CultureInfo.InvariantCulture))
?.Pipe(TimeSpan.FromSeconds);
// If this was the last request available before hitting the rate limit,
// wait out the reset time so that future requests can succeed.
@@ -161,7 +161,7 @@ public class DiscordClient(
$"""
Request to '{url}' failed: {response
.StatusCode.ToString()
.ToSpaceSeparatedWords()
.SeparateWords(' ')
.ToLowerInvariant()}.
Response content: {await response.Content.ReadAsStringAsync(
cancellationToken
@@ -194,6 +194,23 @@ public class DiscordClient(
return Application.Parse(response);
}
private async ValueTask EnsureMessageContentIntentAsync(
CancellationToken cancellationToken = default
)
{
if (await ResolveTokenKindAsync(cancellationToken) != TokenKind.Bot)
return;
var application = await GetApplicationAsync(cancellationToken);
if (application.IsMessageContentIntentEnabled)
return;
throw new DiscordChatExporterException(
"Provided bot account is missing the MESSAGE_CONTENT privileged intent.",
true
);
}
public async ValueTask<User?> TryGetUserAsync(
Snowflake userId,
CancellationToken cancellationToken = default
@@ -371,21 +388,40 @@ public class DiscordClient(
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
try
{
var parent = parentId is not null
? await GetChannelAsync(parentId.Value, cancellationToken)
: null;
return Channel.Parse(response, parent);
}
// It's possible for the parent channel to be inaccessible, despite the
// child channel being accessible.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1108
catch (DiscordChatExporterException)
var parent = parentId is not null
? await TryGetChannelAsync(parentId.Value, cancellationToken)
: null;
return Channel.Parse(response, parent);
}
public async ValueTask<Channel?> TryGetChannelAsync(
Snowflake channelId,
CancellationToken cancellationToken = default
)
{
var response = await TryGetJsonResponseAsync($"channels/{channelId}", cancellationToken);
if (response is null)
return null;
var parentId = response
.Value.GetPropertyOrNull("parent_id")
?.GetNonWhiteSpaceStringOrNull()
?.Pipe(Snowflake.Parse);
Channel? parent = null;
if (parentId is not null)
{
return Channel.Parse(response);
// It's possible for the parent channel to be inaccessible, despite the
// child channel being accessible.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1108
parent = await TryGetChannelAsync(parentId.Value, cancellationToken);
}
return Channel.Parse(response.Value, parent);
}
public async IAsyncEnumerable<Channel> GetChannelThreadsAsync(
@@ -410,6 +446,11 @@ public class DiscordClient(
.Where(c => before is null || c.MayHaveMessagesBefore(before.Value))
.ToArray();
// Track yielded thread IDs to avoid duplicates that can occur when a thread transitions
// from active to archived between the two separate API calls used to fetch threads.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1433
var seenThreadIds = new HashSet<Snowflake>();
// User accounts can only fetch threads using the search endpoint
if (await ResolveTokenKindAsync(cancellationToken) == TokenKind.User)
{
@@ -453,7 +494,9 @@ public class DiscordClient(
break;
}
yield return thread;
if (seenThreadIds.Add(thread.Id))
yield return thread;
currentOffset++;
}
@@ -492,7 +535,12 @@ public class DiscordClient(
.Pipe(parentsById.GetValueOrDefault);
if (filteredChannels.Contains(parent))
yield return Channel.Parse(threadJson, parent);
{
var thread = Channel.Parse(threadJson, parent);
if (seenThreadIds.Add(thread.Id))
yield return thread;
}
}
}
@@ -528,12 +576,14 @@ public class DiscordClient(
)
{
var thread = Channel.Parse(threadJson, channel);
yield return thread;
currentBefore = threadJson
.GetProperty("thread_metadata")
.GetProperty("archive_timestamp")
.GetString();
if (seenThreadIds.Add(thread.Id))
yield return thread;
}
if (!response.Value.GetProperty("has_more").GetBoolean())
@@ -545,6 +595,24 @@ public class DiscordClient(
}
}
private async ValueTask<Message?> TryGetFirstMessageAsync(
Snowflake channelId,
Snowflake? after = null,
CancellationToken cancellationToken = default
)
{
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("limit", "1")
.SetQueryParameter("after", (after ?? Snowflake.Zero).ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
var message = response.EnumerateArray().Select(Message.Parse).FirstOrDefault();
return message;
}
private async ValueTask<Message?> TryGetLastMessageAsync(
Snowflake channelId,
Snowflake? before = null,
@@ -603,22 +671,10 @@ public class DiscordClient(
yield break;
// If all messages are empty, make sure that it's not because the bot account doesn't
// have the Message Content Intent enabled.
// have the MESSAGE_CONTENT intent enabled.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1106#issuecomment-1741548959
if (
messages.All(m => m.IsEmpty)
&& await ResolveTokenKindAsync(cancellationToken) == TokenKind.Bot
)
{
var application = await GetApplicationAsync(cancellationToken);
if (!application.IsMessageContentIntentEnabled)
{
throw new DiscordChatExporterException(
"Provided bot account does not have the Message Content Intent enabled.",
true
);
}
}
if (messages.All(m => m.IsEmpty))
await EnsureMessageContentIntentAsync(cancellationToken);
foreach (var message in messages)
{
@@ -651,6 +707,75 @@ public class DiscordClient(
}
}
public async IAsyncEnumerable<Message> GetMessagesInReverseAsync(
Snowflake channelId,
Snowflake? after = null,
Snowflake? before = null,
IProgress<Percentage>? progress = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default
)
{
// Get the first message in the specified range, so we can later calculate the
// progress based on the difference between message timestamps.
// Snapshotting is not necessary here because new messages can't appear in the past.
var firstMessage = await TryGetFirstMessageAsync(channelId, after, cancellationToken);
if (firstMessage is null || firstMessage.Timestamp > before?.ToDate())
yield break;
// Keep track of the last message in range in order to calculate the progress
var lastMessage = default(Message);
var currentBefore = before;
while (true)
{
var url = new UrlBuilder()
.SetPath($"channels/{channelId}/messages")
.SetQueryParameter("limit", "100")
.SetQueryParameter("before", currentBefore?.ToString())
.Build();
var response = await GetJsonResponseAsync(url, cancellationToken);
var messages = response.EnumerateArray().Select(Message.Parse).ToArray();
// Break if there are no messages (can happen if messages are deleted during execution)
if (!messages.Any())
yield break;
// If all messages are empty, make sure that it's not because the bot account doesn't
// have the MESSAGE_CONTENT intent enabled.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1106#issuecomment-1741548959
if (messages.All(m => m.IsEmpty))
await EnsureMessageContentIntentAsync(cancellationToken);
foreach (var message in messages)
{
lastMessage ??= message;
// Report progress based on timestamps
if (progress is not null)
{
var exportedDuration = (lastMessage.Timestamp - message.Timestamp).Duration();
var totalDuration = (lastMessage.Timestamp - firstMessage.Timestamp).Duration();
progress.Report(
Percentage.FromFraction(
// Avoid division by zero if all messages have the exact same timestamp
// (which happens when there's only one message in the channel)
totalDuration > TimeSpan.Zero
? exportedDuration / totalDuration
: 1
)
);
}
yield return message;
}
currentBefore = messages.Last().Id;
}
}
public async IAsyncEnumerable<User> GetMessageReactionsAsync(
Snowflake channelId,
Snowflake messageId,
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;
using System.IO.Compression;
using System.Linq;
@@ -1,3 +1,3 @@
namespace DiscordChatExporter.Core.Discord.Dump;
namespace DiscordChatExporter.Core.Discord.Dump;
public record DataDumpChannel(Snowflake Id, string Name);
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Discord;
@@ -1,4 +1,4 @@
using System;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Core.Discord;
public enum TokenKind
{
@@ -1,14 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<PackageReference Include="AngleSharp" Version="1.4.0" />
<PackageReference Include="AsyncKeyedLock" Version="8.0.1" />
<PackageReference Include="CSharpier.MsBuild" Version="1.2.5" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Polly" Version="8.6.5" />
<PackageReference Include="RazorBlade" Version="0.11.0" />
<PackageReference Include="Superpower" Version="3.1.0" />
<PackageReference Include="WebMarkupMin.Core" Version="2.20.1" />
<PackageReference Include="YoutubeExplode" Version="6.5.6" />
<PackageReference Include="AngleSharp" />
<PackageReference Include="AsyncKeyedLock" />
<PackageReference Include="CSharpier.MsBuild" PrivateAssets="all" />
<PackageReference Include="Gress" />
<PackageReference Include="JsonExtensions" />
<PackageReference Include="Polly" />
<PackageReference Include="PowerKit" PrivateAssets="all" />
<PackageReference Include="RazorBlade" />
<PackageReference Include="Superpower" />
<PackageReference Include="WebMarkupMin.Core" />
<PackageReference Include="YoutubeExplode" />
</ItemGroup>
</Project>
@@ -1,4 +1,4 @@
using System;
using System;
namespace DiscordChatExporter.Core.Exceptions;

Some files were not shown because too many files have changed in this diff Show More