Compare commits

..

60 Commits

Author SHA1 Message Date
Tyrrrz c3abeadf01 Update version 2023-06-09 01:20:55 +03:00
Tyrrrz 86ee9c8d5f Fix deploy in workflow 2023-06-09 01:20:47 +03:00
Tyrrrz c99355dec3 Consistency in parsing 2023-06-09 01:20:39 +03:00
Tyrrrz 11d34109a7 Extract thread status and improve CLI output 2023-06-09 01:20:32 +03:00
Tyrrrz 609ca0fc0d Simplify guide for extracting channel IDs 2023-06-09 01:19:36 +03:00
Tyrrrz 6cadb19cc8 Update NuGet packages 2023-06-09 01:19:12 +03:00
Tyrrrz d02649773e Fix tests 2023-06-08 16:53:03 +03:00
Tyrrrz 18d9c189b7 Update docs with more information on message filter caveats in CLI
Closes #1045
2023-06-08 16:48:32 +03:00
Tyrrrz 3d2d197904 Introduce the foundation for the new username system 2023-06-08 16:37:05 +03:00
Tyrrrz bf0d8ab31e Rename markdown Header to Heading to avoid ambiguity and for consistency with MD/HTML standards 2023-06-02 13:59:39 +03:00
Tyrrrz 30b447d2ae Update NuGet packages 2023-05-31 22:15:24 +03:00
Tyrrrz 3d047321cf Parse YouTube video IDs natively 2023-05-31 22:14:58 +03:00
Tyrrrz 6d787c4071 Collect source information in tests 2023-05-30 01:31:46 +03:00
Tyrrrz d779ed9bc8 Blur the token in GUI when the control is out of focus 2023-05-29 19:01:49 +03:00
Tyrrrz 3319b4734d Use event data to reference repo info in workflows 2023-05-27 23:49:05 +03:00
Tyrrrz e052241713 Simplify workflows 2023-05-27 23:32:24 +03:00
Tyrrrz 2b7f1ed4ed Use granular permissions for GitHub Actions 2023-05-27 22:42:04 +03:00
Tyrrrz 03784f0d7c Update favicon 2023-05-26 20:40:45 +03:00
Tyrrrz e7310274ba Clean up tests 2023-05-25 18:51:49 +03:00
Tyrrrz 4eab781450 Update NuGet packages 2023-05-25 09:13:21 +03:00
Tyrrrz 6582e26cd3 Increase padding in output to accomodate longer snowflakes 2023-05-24 21:54:51 +03:00
Tyrrrz c48899e0b1 Refactor 2023-05-24 21:51:24 +03:00
William Unsworth 41b4e47d0e Use alternative endpoint to retrieve threads when requested by a bot (#1037)
Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2023-05-24 19:02:34 +03:00
Tyrrrz 6bbde4ccdc Add support for lists in markdown 2023-05-22 14:14:17 +03:00
Tyrrrz 469a731892 Add support for headers in markdown 2023-05-22 11:05:14 +03:00
Tyrrrz d8315c7827 Rename AsyncExtensions to AsyncCollectionExtensions 2023-05-22 10:08:38 +03:00
Tyrrrz a4111c184e Better test naming 2023-05-20 07:32:21 +03:00
Tyrrrz c945c98f75 Update NuGet package 2023-05-20 07:11:16 +03:00
Tyrrrz 31c7ae9312 Refactor 2023-05-20 07:09:19 +03:00
Tyrrrz 03c5c1bc5e Clean up 2023-05-19 14:49:31 +03:00
Tyrrrz 0ae9062a30 Fix typo and refactor 2023-05-19 02:13:19 +03:00
William Unsworth 25caf04445 Preliminary support for threads (#1032) 2023-05-19 01:46:37 +03:00
Tyrrrz 9048557b17 Short-circuit if trying to export an empty channel or a channel with mismatching boundaries
Closes #1024
2023-05-13 21:45:15 +03:00
Tyrrrz faad994332 Add docker badge 2023-05-11 18:10:54 +03:00
Tyrrrz d7537425f8 Use tar as output for docker to avoid unsupported file paths 2023-05-11 16:17:30 +03:00
Tyrrrz e261edde8b Re-add the Docker build job on GitHub Actions 2023-05-11 16:07:06 +03:00
Tyrrrz 72bb886a5e More explicit names 2023-05-11 01:59:36 +03:00
Tyrrrz 9428d5ed56 Prettify workflow files 2023-05-11 01:06:58 +03:00
Tyrrrz b8b1fdb1d1 Make push work in docker workflow 2023-05-11 00:39:09 +03:00
Tyrrrz 168b3c2f31 Fix docker workflow issues 2023-05-11 00:27:22 +03:00
Tyrrrz 9575fd11b5 Update docker workflow 2023-05-11 00:20:29 +03:00
Tyrrrz a332bda07d Make dockerfile more platform-agnostic 2023-05-11 00:06:30 +03:00
Tyrrrz c3f181c9e0 Clean up the docker file 2023-05-10 22:02:50 +03:00
Tyrrrz b9cd40f211 Fix notify in CI 2023-05-02 19:20:23 +03:00
Tyrrrz ab933a7240 Rename TitleLinkNodeMatcher to MaskedLinkNodeMatcher to align with Discord's own terminology 2023-04-30 20:05:31 +03:00
Tyrrrz a9fc439cc5 Update NuGet packages 2023-04-28 17:20:04 +03:00
Tyrrrz d4f5fc5efc Update NuGet packages 2023-04-24 19:39:32 +03:00
Tyrrrz 505330e117 Consistency 2023-04-14 21:04:21 +03:00
Tyrrrz 36db99b5cf Fix workflows 2023-04-06 11:01:45 +03:00
Tyrrrz a4110a362f Use my own GitHub actions 2023-04-05 19:17:43 +03:00
Tyrrrz 0ab8bd67c4 Clean up 2023-04-02 11:41:43 +03:00
Jamie Dubs 31952a109e Fix broken links to the 'Scheduling-*' guides (#1020) 2023-03-23 21:19:04 +02:00
Tyrrrz 2e1636e6c9 Re-implement changes in the previous commit 2023-03-23 13:23:32 +02:00
Tyrrrz 20e782e6ed Revert "Add env variable fallback for --fuck-russia"
This reverts commit ebb838bb81.

Closes #1019
2023-03-23 02:59:43 +02:00
Tyrrrz ebb838bb81 Add env variable fallback for --fuck-russia 2023-03-15 20:44:24 +02:00
Tyrrrz 966627f289 Update version 2023-03-10 16:35:13 +02:00
Tyrrrz 4ec00d0ec8 Clean up CSS 2023-03-10 16:33:56 +02:00
Tyrrrz 8bc1bcaf16 Render linked videos as embeds in HTML
Related to #1012
2023-03-10 16:17:35 +02:00
Tyrrrz 8f40acdda7 Load mentions for users that left the guild 2023-03-08 12:32:59 +02:00
Mark Cilia Vincenti bbe18368fa Update to AsyncKeyedLock 6.2.0 (#1003) 2023-02-25 20:30:00 +02:00
94 changed files with 1213 additions and 731 deletions
+16 -4
View File
@@ -2,7 +2,7 @@
You can use a special notation to filter messages that you want to have included in an export. The notation syntax is designed to mimic Discord's search query syntax, but with additional capabilities.
To configure a filter, specify it in advanced export parameters when using the GUI or by passing the `--filter` option when using the CLI.
To configure a filter, specify it in advanced export parameters when using the GUI or by passing the `--filter` option when using the CLI. For the CLI version, see also [caveats](#cli-caveats).
## Examples
@@ -84,10 +84,22 @@ from:Tyrrrz | from:"96-LB"
from:96\-LB
```
## CLI Caveat
## CLI Caveats
Negated filters (those that start with `-`) may cause parsing issues when using the CLI version of DiscordChatExporter. To avoid this, use the tilde (`~`) character instead of the dash (`-`):
In most cases, you will need to enclose your filter in quotes (`"`) to escape characters that may have special meaning in your shell:
```
DiscordChatExporter.Cli export [...] --filter "from:Tyrrrz has:image"
```
If you need to include quotes inside the filter itself as well, use single quotes (`'`) for those instead:
```
DiscordChatExporter.Cli export [...] --filter "from:Tyrrrz 'hello world'"
```
Additionally, negated filters (those that start with `-`) may cause parsing issues even when enclosed in quotes. To avoid this, use the tilde (`~`) character instead of the dash (`-`):
```
DiscordChatExporter.Cli export [...] --filter ~from:Tyrrrz
```
```
+3 -3
View File
@@ -14,9 +14,9 @@
- [How to get Token and Channel IDs](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Token-and-IDs.md)
- [How to use message filters](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Message-filters.md)
- Export scheduling with CLI:
- [Windows](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/scheduling-windows.md)
- [macOS](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/scheduling-MacOS.md)
- [Linux](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/scheduling-Linux.md)
- [Windows](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Scheduling-Windows.md)
- [macOS](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Scheduling-MacOS.md)
- [Linux](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Scheduling-Linux.md)
## Video tutorial
+2 -22
View File
@@ -234,29 +234,9 @@ Prerequisite step: Navigate to [discord.com](https://discord.com) and login.
---
## How to get a Server ID or a Server Channel ID
## How to get a Server ID or a Channel ID
1. Open Discord Settings
2. Go to the **Advanced** section
3. Enable **Developer Mode**
4. Right-click on the desired server or channel and click Copy ID
## How to get a Direct Message Channel ID
1. Click the three dots next to any message in the channel you want to export.
2. Paste the link into a text editor.
3. Copy the text in the highlighted part of the link.
> E.g. in https//discord.com/channels/@me/**189716987098470**/0985709387059874, you should copy **189716987098470**
Make sure you're not copying the user's ID.
**Alternatively, you can also:**
1. Open the desired direct message channel
2. Press <kbd>Ctrl</kbd>+<kbd>Shift</kbd>+<kbd>I</kbd> (<kbd>⌥</kbd>+<kbd>⌘</kbd>+<kbd>I</kbd> on macOS) on Discord to show developer tools
3. Navigate to the `Console` tab
4. Type `window.location.href` and press Enter
5. Copy the first long sequence of numbers inside the URL
> E.g. in https//discord.com/channels/@me/**84289740160**, you should copy **84289740160**
4. Right-click on the desired server or channel and click Copy Server ID or Copy Channel ID
+51 -20
View File
@@ -5,51 +5,82 @@ on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
permissions:
actions: write
contents: read
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Build container
run: docker build -f DiscordChatExporter.Cli.dockerfile .
- name: Install Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Build image
run: >
docker buildx build
--file DiscordChatExporter.Cli.dockerfile
--platform linux/amd64,linux/arm64
--output type=tar,dest=DiscordChatExporter.Cli.Docker.tar
.
- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: DiscordChatExporter.Cli.Docker
path: DiscordChatExporter.Cli.Docker.tar
deploy-latest:
if: ${{ github.event_name == 'push' && github.ref_type == 'branch' && github.ref_name == 'master' }}
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to DockerHub
run: docker login -u tyrrrz -p ${{ secrets.DOCKER_TOKEN }}
run: >
echo ${{ secrets.DOCKER_TOKEN }} |
docker login --username tyrrrz --password-stdin
- name: Build container
run: docker build -f DiscordChatExporter.Cli.dockerfile -t tyrrrz/discordchatexporter:latest .
- name: Push container
run: docker push tyrrrz/discordchatexporter:latest
- name: Build & push image
run: >
docker buildx build
--file DiscordChatExporter.Cli.dockerfile
--platform linux/amd64,linux/arm64
--push
--tag tyrrrz/discordchatexporter:latest
.
deploy-stable:
if: ${{ github.event_name == 'push' && github.ref_type == 'tag' }}
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Get release version
id: get_version
uses: dawidd6/action-get-tag@v1
- name: Install Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to DockerHub
run: docker login -u tyrrrz -p ${{ secrets.DOCKER_TOKEN }}
run: >
echo ${{ secrets.DOCKER_TOKEN }} |
docker login --username tyrrrz --password-stdin
- name: Build container
run: docker build -f DiscordChatExporter.Cli.dockerfile -t tyrrrz/discordchatexporter:stable -t tyrrrz/discordchatexporter:${{ steps.get_version.outputs.tag }} .
- name: Push container (stable)
run: docker push tyrrrz/discordchatexporter:stable
- name: Push container (${{ steps.get_version.outputs.tag }})
run: docker push tyrrrz/discordchatexporter:${{ steps.get_version.outputs.tag }}
- name: Build & push image
run: >
docker buildx build
--file DiscordChatExporter.Cli.dockerfile
--platform linux/amd64,linux/arm64
--push
--tag tyrrrz/discordchatexporter:stable
--tag tyrrrz/discordchatexporter:${{ github.ref_name }}
.
+52 -26
View File
@@ -5,6 +5,8 @@ on: [push, pull_request]
jobs:
test:
runs-on: windows-latest
permissions:
contents: read
env:
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
@@ -23,7 +25,14 @@ jobs:
- name: Run tests
# Tests need access to secrets, so we can't run them against PRs because of limited trust
if: ${{ github.event_name != 'pull_request' }}
run: dotnet test --configuration Release --logger GitHubActions --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
run: >
dotnet test
--configuration Release
--logger "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
--collect:"XPlat Code Coverage"
--
RunConfiguration.CollectSourceInformation=true
DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
env:
DISCORD_TOKEN: ${{ secrets.DISCORD_TOKEN }}
@@ -32,9 +41,12 @@ jobs:
with:
token: ${{ secrets.CODECOV_TOKEN }}
package:
pack:
needs: test
runs-on: windows-latest
permissions:
actions: write
contents: read
env:
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
@@ -51,27 +63,36 @@ jobs:
dotnet-version: 7.0.x
- name: Publish (CLI)
run: dotnet publish DiscordChatExporter.Cli/ -o DiscordChatExporter.Cli/bin/Publish/ --configuration Release
run: >
dotnet publish DiscordChatExporter.Cli
--output DiscordChatExporter.Cli/publish/
--configuration Release
- name: Publish (GUI)
run: dotnet publish DiscordChatExporter.Gui/ -o DiscordChatExporter.Gui/bin/Publish/ --configuration Release
run: >
dotnet publish DiscordChatExporter.Gui
--output DiscordChatExporter.Gui/publish/
--configuration Release
- name: Upload artifacts (CLI)
uses: actions/upload-artifact@v3
with:
name: DiscordChatExporter.Cli
path: DiscordChatExporter.Cli/bin/Publish/
path: DiscordChatExporter.Cli/publish/
- name: Upload artifacts (GUI)
uses: actions/upload-artifact@v3
with:
name: DiscordChatExporter
path: DiscordChatExporter.Gui/bin/Publish/
path: DiscordChatExporter.Gui/publish/
deploy:
if: ${{ github.event_name == 'push' && github.ref_type == 'tag' }}
needs: package
needs: pack
runs-on: ubuntu-latest
permissions:
actions: read
contents: write
steps:
- name: Download artifacts (CLI)
@@ -87,43 +108,48 @@ jobs:
path: DiscordChatExporter.Gui
- name: Create package (CLI)
run: Compress-Archive -Path DiscordChatExporter.Cli/* -DestinationPath DiscordChatExporter.Cli.zip -Force
shell: pwsh
run: >
Compress-Archive
-Path DiscordChatExporter.Cli/*
-DestinationPath DiscordChatExporter.Cli.zip
-Force
- name: Create package (GUI)
run: Compress-Archive -Path DiscordChatExporter.Gui/* -DestinationPath DiscordChatExporter.zip -Force
shell: pwsh
run: >
Compress-Archive
-Path DiscordChatExporter.Gui/*
-DestinationPath DiscordChatExporter.zip
-Force
- name: Create release
uses: softprops/action-gh-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
files: |
DiscordChatExporter.Cli.zip
DiscordChatExporter.zip
body: |
[Changelog](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/Changelog.md)
run: >
gh release create "${{ github.ref_name }}"
"DiscordChatExporter.Cli.zip"
"DiscordChatExporter.zip"
--repo "${{ github.event.repository.full_name }}"
--title "${{ github.ref_name }}"
--notes "[Changelog](${{ github.event.repository.html_url }}/blob/${{ github.ref_name }}/Changelog.md)"
--verify-tag
notify:
needs: deploy
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Get release version
id: get_version
uses: dawidd6/action-get-tag@v1
- name: Notify Discord
uses: satak/webrequest-action@v1.2.4
uses: tyrrrz/action-http-request@v1
with:
url: ${{ secrets.DISCORD_WEBHOOK }}
method: POST
headers: |
Content-Type: application/json; charset=UTF-8
body: |
{
"ContentType": "application/json; charset=UTF-8"
}
payload: |
{
"content": "**DiscordChatExporter** new version released!\nVersion: `${{ steps.get_version.outputs.tag }}`\nChangelog: <https://github.com/Tyrrrz/DiscordChatExporter/blob/${{ steps.get_version.outputs.tag }}/Changelog.md>"
"content": "**${{ github.event.repository.name }}** new version released!\nVersion: `${{ github.ref_name }}`\nChangelog: <${{ github.event.repository.html_url }}/blob/${{ github.ref_name }}/Changelog.md>"
}
+20 -1
View File
@@ -1,8 +1,27 @@
# Changelog
## v2.40 (08-Jun-2023)
- General changes:
- Added support for users that don't have discriminators.
- Added a precondition check that verifies whether the requested channel contains any messages for the specified period, before starting the export process. This check works by comparing the export bounds against the ID of the last message in the channel, which is provided by the Discord API as part of the channel metadata. When exporting many channels in batches (e.g. using the `exportall` command), this check may eliminate a significant number of unnecessary API calls and speed up the export process.
- CLI changes:
- Extended the `channels` command with the `--include-threads` option. When this option is provided, the command will include channel threads in the output. (Thanks [@WAUthethird](https://github.com/WAUthethird))
- Simplified the information in the `guide` command related to the extraction of DM channel IDs. Discord now provides a native way of copying IDs of DM channels, so the previous devtools-based workaround is no longer necessary.
- GUI changes:
- Added a blur effect to the token text box, which activates when the control loses focus. This is done to reduce the risk of accidentally exposing the token when taking screenshots.
- HTML format changes:
- Added support for the header markdown syntax.
- Added support for the list markdown syntax.
## v2.39.1 (10-Mar-2023)
- Fixed an issue where messages containing mentions of users that left the server (but haven't deleted their account) were incorrectly rendered as `@Unknown`.
- Fixed an issue where messages containing video links were not rendered correctly in the HTML export. Now, they are rendered using a video player and the referenced assets are pulled if the "download assets" option is enabled.
## v2.39 (25-Feb-2023)
- Added an option to specify the directory path for downloaded assets. This can be used, in combination with the "reuse assets" option, to have a shared asset directory for multiple exports. In GUI, this option can be found in the advanced setion of the export dialog. In CLI, it's available as `--media-dir <path>`. (Thanks [@96-LB](https://github.com/96-LB))
- Added an option to specify the directory path for downloaded assets. This can be used, in combination with the "reuse assets" option, to have a shared asset directory for multiple exports. In GUI, this option can be found in the advanced section of the export dialog. In CLI, it's available as `--media-dir <path>`. (Thanks [@96-LB](https://github.com/96-LB))
- Added support for server-level user avatars. If a user has a custom avatar set on the server, it will be rendered instead of their main avatar.
- CLI changes:
- Extended the `exportall` command with the `--data-package` option. You can use this option to provide the path to your personal data package (`package.zip` file), requested from Discord. When provided, the command will pull channels listed in the package instead of using the Discord API to query them. This can be particularly useful if you want to export DM channels that have been closed, as the personal data package should still contain their IDs.
+1 -1
View File
@@ -2,7 +2,7 @@
<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<Version>2.39</Version>
<Version>2.40</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Oleksii Holub</Copyright>
<LangVersion>preview</LangVersion>
@@ -12,18 +12,18 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="AngleSharp" Version="1.0.1" />
<PackageReference Include="FluentAssertions" Version="6.10.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="2.0.1" PrivateAssets="all" />
<PackageReference Include="AngleSharp" Version="1.0.3" />
<PackageReference Include="FluentAssertions" Version="6.11.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="2.3.2" PrivateAssets="all" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="7.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="7.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="7.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.5.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.2" />
<PackageReference Include="ReflectionMagic" Version="4.1.0" />
<PackageReference Include="xunit" Version="2.4.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" PrivateAssets="all" />
<PackageReference Include="coverlet.collector" Version="3.2.0" PrivateAssets="all" />
<PackageReference Include="coverlet.collector" Version="6.0.0" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
@@ -51,7 +51,7 @@ public static class ExportWrapper
// Lock separately for each channel and format
using (await Locker.LockAsync(filePath))
{
// Perform export only if it hasn't been done before
// Perform the export only if it hasn't been done before
if (!File.Exists(filePath))
{
await new ExportChannelsCommand
@@ -94,14 +94,13 @@ public static class ExportWrapper
public static async ValueTask<IElement> GetMessageAsHtmlAsync(Snowflake channelId, Snowflake messageId)
{
var message = (await GetMessagesAsHtmlAsync(channelId))
.SingleOrDefault(e =>
string.Equals(
e.GetAttribute("data-message-id"),
messageId.ToString(),
StringComparison.OrdinalIgnoreCase
)
);
var message = (await GetMessagesAsHtmlAsync(channelId)).SingleOrDefault(e =>
string.Equals(
e.GetAttribute("data-message-id"),
messageId.ToString(),
StringComparison.OrdinalIgnoreCase
)
);
if (message is null)
{
@@ -115,14 +114,13 @@ public static class ExportWrapper
public static async ValueTask<JsonElement> GetMessageAsJsonAsync(Snowflake channelId, Snowflake messageId)
{
var message = (await GetMessagesAsJsonAsync(channelId))
.SingleOrDefault(j =>
string.Equals(
j.GetProperty("id").GetString(),
messageId.ToString(),
StringComparison.OrdinalIgnoreCase
)
);
var message = (await GetMessagesAsJsonAsync(channelId)).SingleOrDefault(j =>
string.Equals(
j.GetProperty("id").GetString(),
messageId.ToString(),
StringComparison.OrdinalIgnoreCase
)
);
if (message.ValueKind == JsonValueKind.Undefined)
{
@@ -8,14 +8,14 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class CsvContentSpecs
{
[Fact]
public async Task Messages_are_exported_correctly()
public async Task I_can_export_a_channel_in_the_CSV_format()
{
// Act
var document = await ExportWrapper.ExportAsCsvAsync(ChannelIds.DateRangeTestCases);
// Assert
document.Should().ContainAll(
"Tyrrrz#5447",
"tyrrrz",
"Hello world",
"Goodbye world",
"Foo bar",
@@ -17,7 +17,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class DateRangeSpecs
{
[Fact]
public async Task Messages_filtered_after_specific_date_only_include_messages_sent_after_that_date()
public async Task I_can_filter_the_export_to_only_include_messages_sent_after_the_specified_date()
{
// Arrange
var after = new DateTimeOffset(2021, 07, 24, 0, 0, 0, TimeSpan.Zero);
@@ -61,7 +61,7 @@ public class DateRangeSpecs
}
[Fact]
public async Task Messages_filtered_before_specific_date_only_include_messages_sent_before_that_date()
public async Task I_can_filter_the_export_to_only_include_messages_sent_before_the_specified_date()
{
// Arrange
var before = new DateTimeOffset(2021, 07, 24, 0, 0, 0, TimeSpan.Zero);
@@ -103,7 +103,7 @@ public class DateRangeSpecs
}
[Fact]
public async Task Messages_filtered_between_specific_dates_only_include_messages_sent_between_those_dates()
public async Task I_can_filter_the_export_to_only_include_messages_sent_between_the_specified_dates()
{
// Arrange
var after = new DateTimeOffset(2021, 07, 24, 0, 0, 0, TimeSpan.Zero);
@@ -16,7 +16,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class FilterSpecs
{
[Fact]
public async Task Messages_filtered_by_text_only_include_messages_that_contain_that_text()
public async Task I_can_filter_the_export_to_only_include_messages_that_contain_the_specified_text()
{
// Arrange
using var file = TempFile.Create();
@@ -42,7 +42,7 @@ public class FilterSpecs
}
[Fact]
public async Task Messages_filtered_by_author_only_include_messages_sent_by_that_author()
public async Task I_can_filter_the_export_to_only_include_messages_that_were_sent_by_the_specified_author()
{
// Arrange
using var file = TempFile.Create();
@@ -64,11 +64,11 @@ public class FilterSpecs
.EnumerateArray()
.Select(j => j.GetProperty("author").GetProperty("name").GetString())
.Should()
.AllBe("Tyrrrz");
.AllBe("tyrrrz");
}
[Fact]
public async Task Messages_filtered_by_content_only_include_messages_that_have_that_content()
public async Task I_can_filter_the_export_to_only_include_messages_that_contain_the_specified_content()
{
// Arrange
using var file = TempFile.Create();
@@ -94,7 +94,7 @@ public class FilterSpecs
}
[Fact]
public async Task Messages_filtered_by_pin_only_include_messages_that_have_been_pinned()
public async Task I_can_filter_the_export_to_only_include_messages_that_have_been_pinned()
{
// Arrange
using var file = TempFile.Create();
@@ -120,7 +120,7 @@ public class FilterSpecs
}
[Fact]
public async Task Messages_filtered_by_mention_only_include_messages_that_have_that_mention()
public async Task I_can_filter_the_export_to_only_include_messages_that_contain_the_specified_mention()
{
// Arrange
using var file = TempFile.Create();
@@ -11,7 +11,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlAttachmentSpecs
{
[Fact]
public async Task Message_with_a_generic_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_generic_attachment()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -36,7 +36,7 @@ public class HtmlAttachmentSpecs
}
[Fact]
public async Task Message_with_an_image_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_an_image_attachment()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -57,7 +57,7 @@ public class HtmlAttachmentSpecs
}
[Fact]
public async Task Message_with_a_video_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_video_attachment()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/333
@@ -77,7 +77,7 @@ public class HtmlAttachmentSpecs
}
[Fact]
public async Task Message_with_an_audio_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_an_audio_attachment()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/333
@@ -2,7 +2,6 @@
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using FluentAssertions;
using Xunit;
@@ -11,7 +10,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlContentSpecs
{
[Fact]
public async Task Messages_are_exported_correctly()
public async Task I_can_export_a_channel_in_the_HTML_format()
{
// Act
var messages = await ExportWrapper.GetMessagesAsHtmlAsync(ChannelIds.DateRangeTestCases);
@@ -39,21 +38,4 @@ public class HtmlContentSpecs
"Yeet"
);
}
[Fact]
public async Task Messages_cross_posted_from_other_guilds_are_rendered_with_the_server_tag()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/633
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.ReplyTestCases,
Snowflake.Parse("1072165330853576876")
);
// Assert
message.Text().Should().Contain("This is a test message from an announcement channel on another server");
message.Text().Should().Contain("SERVER");
message.QuerySelector(".chatlog__reply-link").Should().BeNull();
}
}
@@ -3,6 +3,7 @@ using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils.Extensions;
using FluentAssertions;
using Xunit;
@@ -11,7 +12,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlEmbedSpecs
{
[Fact]
public async Task Message_with_an_embed_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_rich_embed()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -32,7 +33,7 @@ public class HtmlEmbedSpecs
}
[Fact]
public async Task Message_with_an_image_link_is_rendered_with_an_image_embed()
public async Task I_can_export_a_channel_that_contains_a_message_with_an_image_embed()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/537
@@ -46,13 +47,14 @@ public class HtmlEmbedSpecs
message
.QuerySelectorAll("img")
.Select(e => e.GetAttribute("src"))
.Where(s => s?.EndsWith("i.redd.it/f8w05ja8s4e61.png") ?? false)
.WhereNotNull()
.Where(s => s.EndsWith("f8w05ja8s4e61.png"))
.Should()
.ContainSingle();
}
[Fact]
public async Task Message_with_an_image_link_and_nothing_else_is_rendered_without_text_content()
public async Task I_can_export_a_channel_that_contains_a_message_with_an_image_embed_and_the_text_is_hidden_if_it_only_contains_the_image_link()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/682
@@ -68,7 +70,26 @@ public class HtmlEmbedSpecs
}
[Fact]
public async Task Message_with_a_GIFV_link_is_rendered_with_a_video_embed()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_video_embed()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.EmbedTestCases,
Snowflake.Parse("1083751036596002856")
);
// Assert
message
.QuerySelectorAll("source")
.Select(e => e.GetAttribute("src"))
.WhereNotNull()
.Where(s => s.EndsWith("i_am_currently_feeling_slight_displeasure_of_what_you_have_just_sent_lqrem.mp4"))
.Should()
.ContainSingle();
}
[Fact]
public async Task I_can_export_a_channel_that_contains_a_message_with_a_GIFV_embed()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -80,13 +101,14 @@ public class HtmlEmbedSpecs
message
.QuerySelectorAll("source")
.Select(e => e.GetAttribute("src"))
.Where(s => s?.EndsWith("media.tenor.com/DDAJeW6BQKkAAAPo/tooncasm-test-copy.mp4") ?? false)
.WhereNotNull()
.Where(s => s.EndsWith("tooncasm-test-copy.mp4"))
.Should()
.ContainSingle();
}
[Fact]
public async Task Message_with_a_GIFV_link_and_nothing_else_is_rendered_without_text_content()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_GIFV_embed_and_the_text_is_hidden_if_it_only_contains_the_video_link()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -100,7 +122,7 @@ public class HtmlEmbedSpecs
}
[Fact]
public async Task Message_with_a_Spotify_track_link_is_rendered_with_a_track_embed()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_Spotify_track_embed()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/657
@@ -116,7 +138,7 @@ public class HtmlEmbedSpecs
}
[Fact]
public async Task Message_with_a_YouTube_video_link_is_rendered_with_a_video_embed()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_YouTube_video_embed()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/570
@@ -132,7 +154,7 @@ public class HtmlEmbedSpecs
}
[Fact]
public async Task Message_with_a_Twitter_post_link_with_multiple_images_is_rendered_as_a_single_embed()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_Twitter_post_embed_that_includes_multiple_images()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/695
@@ -158,7 +180,7 @@ public class HtmlEmbedSpecs
}
[Fact]
public async Task Message_with_a_guild_invite_link_is_rendered_with_a_widget()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_guild_invite()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/649
@@ -15,7 +15,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlGroupingSpecs
{
[Fact]
public async Task Messages_are_grouped_correctly()
public async Task I_can_export_a_channel_and_the_messages_are_grouped_according_to_their_author_and_timestamps()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/152
@@ -12,7 +12,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlMarkdownSpecs
{
[Fact]
public async Task Message_with_a_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -36,7 +36,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_a_short_time_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker_in_the_short_format()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -60,7 +60,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_a_long_time_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker_in_the_long_format()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -84,7 +84,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_a_short_date_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker_in_the_short_date_format()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -108,7 +108,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_a_long_date_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker_in_the_long_date_format()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -132,7 +132,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_a_full_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker_in_the_full_format()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -156,7 +156,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_a_full_long_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker_in_the_full_long_format()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -180,7 +180,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_a_relative_timestamp_is_rendered_as_the_default_timestamp()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_timestamp_marker_in_the_relative_format()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -204,7 +204,7 @@ public class HtmlMarkdownSpecs
}
[Fact]
public async Task Message_with_an_invalid_timestamp_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_an_invalid_timestamp_marker()
{
// Date formatting code relies on the local time zone, so we need to set it to a fixed value
TimeZoneInfoEx.SetLocal(TimeSpan.FromHours(+2));
@@ -10,7 +10,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlMentionSpecs
{
[Fact]
public async Task Message_with_a_user_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_user_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -20,11 +20,11 @@ public class HtmlMentionSpecs
// Assert
message.Text().Should().Contain("User mention: @Tyrrrz");
message.InnerHtml.Should().Contain("Tyrrrz#5447");
message.InnerHtml.Should().Contain("tyrrrz");
}
[Fact]
public async Task Message_with_a_text_channel_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_text_channel_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -37,7 +37,7 @@ public class HtmlMentionSpecs
}
[Fact]
public async Task Message_with_a_voice_channel_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_voice_channel_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -50,7 +50,7 @@ public class HtmlMentionSpecs
}
[Fact]
public async Task Message_with_a_role_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_role_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -10,7 +10,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlReplySpecs
{
[Fact]
public async Task Message_with_a_reply_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_that_replies_to_another_message()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -24,7 +24,7 @@ public class HtmlReplySpecs
}
[Fact]
public async Task Message_with_a_reply_to_a_deleted_message_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_that_replies_to_a_deleted_message()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/645
@@ -42,7 +42,7 @@ public class HtmlReplySpecs
}
[Fact]
public async Task Message_with_a_reply_to_an_empty_message_with_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_that_replies_to_an_empty_message_with_an_attachment()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/634
@@ -58,7 +58,7 @@ public class HtmlReplySpecs
}
[Fact]
public async Task Message_with_a_reply_to_an_interaction_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_that_replies_to_an_interaction()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/569
@@ -71,4 +71,21 @@ public class HtmlReplySpecs
// Assert
message.Text().Should().Contain("used /poll");
}
[Fact]
public async Task I_can_export_a_channel_that_contains_a_message_cross_posted_from_another_guild()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/633
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.ReplyTestCases,
Snowflake.Parse("1072165330853576876")
);
// Assert
message.Text().Should().Contain("This is a test message from an announcement channel on another server");
message.Text().Should().Contain("SERVER");
message.QuerySelector(".chatlog__reply-link").Should().BeNull();
}
}
@@ -9,7 +9,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class HtmlStickerSpecs
{
[Fact]
public async Task Message_with_a_PNG_based_sticker_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_PNG_sticker()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -23,7 +23,7 @@ public class HtmlStickerSpecs
}
[Fact]
public async Task Message_with_a_Lottie_based_sticker_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_Lottie_sticker()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
@@ -10,7 +10,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class JsonAttachmentSpecs
{
[Fact]
public async Task Message_with_a_generic_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_generic_attachment()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -23,15 +23,16 @@ public class JsonAttachmentSpecs
var attachments = message.GetProperty("attachments").EnumerateArray().ToArray();
attachments.Should().HaveCount(1);
attachments.Single().GetProperty("url").GetString().Should().Be(
attachments[0].GetProperty("url").GetString().Should().Be(
"https://cdn.discordapp.com/attachments/885587741654536192/885587844964417596/Test.txt"
);
attachments.Single().GetProperty("fileName").GetString().Should().Be("Test.txt");
attachments.Single().GetProperty("fileSizeBytes").GetInt64().Should().Be(11);
attachments[0].GetProperty("fileName").GetString().Should().Be("Test.txt");
attachments[0].GetProperty("fileSizeBytes").GetInt64().Should().Be(11);
}
[Fact]
public async Task Message_with_an_image_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_an_image_attachment()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -44,15 +45,16 @@ public class JsonAttachmentSpecs
var attachments = message.GetProperty("attachments").EnumerateArray().ToArray();
attachments.Should().HaveCount(1);
attachments.Single().GetProperty("url").GetString().Should().Be(
attachments[0].GetProperty("url").GetString().Should().Be(
"https://cdn.discordapp.com/attachments/885587741654536192/885654862430359613/bird-thumbnail.png"
);
attachments.Single().GetProperty("fileName").GetString().Should().Be("bird-thumbnail.png");
attachments.Single().GetProperty("fileSizeBytes").GetInt64().Should().Be(466335);
attachments[0].GetProperty("fileName").GetString().Should().Be("bird-thumbnail.png");
attachments[0].GetProperty("fileSizeBytes").GetInt64().Should().Be(466335);
}
[Fact]
public async Task Message_with_a_video_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_video_attachment()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -65,15 +67,16 @@ public class JsonAttachmentSpecs
var attachments = message.GetProperty("attachments").EnumerateArray().ToArray();
attachments.Should().HaveCount(1);
attachments.Single().GetProperty("url").GetString().Should().Be(
attachments[0].GetProperty("url").GetString().Should().Be(
"https://cdn.discordapp.com/attachments/885587741654536192/885655761512968233/file_example_MP4_640_3MG.mp4"
);
attachments.Single().GetProperty("fileName").GetString().Should().Be("file_example_MP4_640_3MG.mp4");
attachments.Single().GetProperty("fileSizeBytes").GetInt64().Should().Be(3114374);
attachments[0].GetProperty("fileName").GetString().Should().Be("file_example_MP4_640_3MG.mp4");
attachments[0].GetProperty("fileSizeBytes").GetInt64().Should().Be(3114374);
}
[Fact]
public async Task Message_with_an_audio_attachment_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_an_audio_attachment()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -86,10 +89,11 @@ public class JsonAttachmentSpecs
var attachments = message.GetProperty("attachments").EnumerateArray().ToArray();
attachments.Should().HaveCount(1);
attachments.Single().GetProperty("url").GetString().Should().Be(
attachments[0].GetProperty("url").GetString().Should().Be(
"https://cdn.discordapp.com/attachments/885587741654536192/885656175348187146/file_example_MP3_1MG.mp3"
);
attachments.Single().GetProperty("fileName").GetString().Should().Be("file_example_MP3_1MG.mp3");
attachments.Single().GetProperty("fileSizeBytes").GetInt64().Should().Be(1087849);
attachments[0].GetProperty("fileName").GetString().Should().Be("file_example_MP3_1MG.mp3");
attachments[0].GetProperty("fileSizeBytes").GetInt64().Should().Be(1087849);
}
}
@@ -9,7 +9,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class JsonContentSpecs
{
[Fact]
public async Task Messages_are_exported_correctly()
public async Task I_can_export_a_channel_in_the_JSON_format()
{
// Act
var messages = await ExportWrapper.GetMessagesAsJsonAsync(ChannelIds.DateRangeTestCases);
@@ -10,7 +10,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class JsonEmbedSpecs
{
[Fact]
public async Task Message_with_an_embed_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_rich_embed()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -10,7 +10,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class JsonMentionSpecs
{
[Fact]
public async Task Message_with_a_user_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_user_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -30,7 +30,7 @@ public class JsonMentionSpecs
}
[Fact]
public async Task Message_with_a_text_channel_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_text_channel_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -43,7 +43,7 @@ public class JsonMentionSpecs
}
[Fact]
public async Task Message_with_a_voice_channel_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_voice_channel_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -56,7 +56,7 @@ public class JsonMentionSpecs
}
[Fact]
public async Task Message_with_a_role_mention_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_role_mention()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -10,7 +10,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class JsonStickerSpecs
{
[Fact]
public async Task Message_with_a_PNG_based_sticker_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_PNG_sticker()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -31,7 +31,7 @@ public class JsonStickerSpecs
}
[Fact]
public async Task Message_with_a_Lottie_based_sticker_is_rendered_correctly()
public async Task I_can_export_a_channel_that_contains_a_message_with_a_Lottie_sticker()
{
// Act
var message = await ExportWrapper.GetMessageAsJsonAsync(
@@ -14,7 +14,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class PartitioningSpecs
{
[Fact]
public async Task Messages_partitioned_by_count_are_split_into_a_corresponding_number_of_files()
public async Task I_can_export_a_channel_with_partitioning_based_on_message_count()
{
// Arrange
using var dir = TempDir.Create();
@@ -37,7 +37,7 @@ public class PartitioningSpecs
}
[Fact]
public async Task Messages_partitioned_by_file_size_are_split_into_a_corresponding_number_of_files()
public async Task I_can_export_a_channel_with_partitioning_based_on_file_size()
{
// Arrange
using var dir = TempDir.Create();
@@ -8,14 +8,14 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class PlainTextContentSpecs
{
[Fact]
public async Task Messages_are_exported_correctly()
public async Task I_can_export_a_channel_in_the_TXT_format()
{
// Act
var document = await ExportWrapper.ExportAsPlainTextAsync(ChannelIds.DateRangeTestCases);
// Assert
document.Should().ContainAll(
"Tyrrrz#5447",
"tyrrrz",
"Hello world",
"Goodbye world",
"Foo bar",
@@ -14,7 +14,7 @@ namespace DiscordChatExporter.Cli.Tests.Specs;
public class SelfContainedSpecs
{
[Fact]
public async Task Messages_in_self_contained_export_only_reference_local_file_resources()
public async Task I_can_export_a_channel_and_download_all_referenced_assets()
{
// Arrange
using var dir = TempDir.Create();
+29 -28
View File
@@ -1,44 +1,45 @@
# -- Build
FROM mcr.microsoft.com/dotnet/sdk:7.0-alpine AS build
# Specify the platform here so that we pull the SDK image matching the host platform,
# instead of the target platform specified during build by the `--platform` option.
# Use the .NET 8.0 preview because the `--arch` option is only available in that version.
# https://github.com/dotnet/dotnet-docker/issues/4388#issuecomment-1459038566
# TODO: switch images to Alpine once .NET 8.0 is released.
# Currently, the correct preview version is only available on Debian.
# https://github.com/dotnet/dotnet-docker/blob/main/samples/selecting-tags.md
FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0-preview AS build
# Expose the target architecture set by the `docker build --platform` option, so that
# we can build the assembly for the correct platform.
ARG TARGETARCH
WORKDIR /build
COPY favicon.ico ./
COPY NuGet.config ./
COPY Directory.Build.props ./
COPY DiscordChatExporter.Core ./DiscordChatExporter.Core
COPY DiscordChatExporter.Cli ./DiscordChatExporter.Cli
COPY favicon.ico .
COPY NuGet.config .
COPY Directory.Build.props .
COPY DiscordChatExporter.Core DiscordChatExporter.Core
COPY DiscordChatExporter.Cli DiscordChatExporter.Cli
# Publish a self-contained assembly so we can use a slimmer runtime image
RUN dotnet publish DiscordChatExporter.Cli \
--configuration Release \
--self-contained \
--use-current-runtime \
--configuration Release \
--output ./publish
--arch $TARGETARCH \
--output publish/
# -- Run
FROM mcr.microsoft.com/dotnet/runtime-deps:7.0-alpine
# Use `runtime-deps` instead of `runtime` because we have a self-contained assembly
FROM --platform=$TARGETPLATFORM mcr.microsoft.com/dotnet/runtime-deps:7.0 AS run
# Alpine dotnet image doesn't include timezone data, which is needed
# for certain date/time operations.
RUN apk add --no-cache tzdata
# Create a non-root user to run the app, so that the output files
# can be accessed by the host.
# Create a non-root user to run the app, so that the output files can be accessed by the host
# https://github.com/Tyrrrz/DiscordChatExporter/issues/851
RUN adduser \
--disabled-password \
--no-create-home \
dce
RUN adduser --disabled-password --no-create-home dce
USER dce
COPY --from=build /build/publish /opt/dce
# Need to keep this as /out for backwards compatibility with documentation.
# A lot of people have this directory mounted in their scripts files, so
# changing it would break existing workflows.
# This directory is exposed to the user for mounting purposes, so it's important that it always
# stays the same for backwards compatibility.
WORKDIR /out
# Add the app directory to PATH so that it's easier to debug using a shell
ENV PATH="$PATH:/opt/dce"
ENTRYPOINT ["DiscordChatExporter.Cli"]
COPY --from=build /build/publish /opt/dce
ENTRYPOINT ["/opt/dce/DiscordChatExporter.Cli"]
@@ -7,6 +7,7 @@ using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Exceptions;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Converters;
using DiscordChatExporter.Cli.Utils.Extensions;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
@@ -29,8 +30,8 @@ public abstract class ExportCommandBase : DiscordCommandBase
'o',
Description =
"Output file or directory path. " +
"If a directory is specified, file names will be generated automatically based on the channel names and other parameters. " +
"Directory path should end with a slash to avoid ambiguity. " +
"Directory path must end with a slash to avoid ambiguity. " +
"If a directory is specified, file names will be generated automatically. " +
"Supports template tokens, see the documentation for more info."
)]
public string OutputPath
@@ -64,13 +65,16 @@ public abstract class ExportCommandBase : DiscordCommandBase
"partition",
'p',
Description =
"Split output into partitions, each limited to this number of messages (e.g. '100') or file size (e.g. '10mb')."
"Split the output into partitions, each limited to the specified " +
"number of messages (e.g. '100') or file size (e.g. '10mb')."
)]
public PartitionLimit PartitionLimit { get; init; } = PartitionLimit.Null;
[CommandOption(
"filter",
Description = "Only include messages that satisfy this filter (e.g. 'from:foo#1234' or 'has:image')."
Description =
"Only include messages that satisfy this filter. " +
"See the documentation for more info."
)]
public MessageFilter MessageFilter { get; init; } = MessageFilter.Null;
@@ -102,7 +106,9 @@ public abstract class ExportCommandBase : DiscordCommandBase
[CommandOption(
"media-dir",
Description = "Download assets to this directory. If not specified, the asset directory path will be derived from the output path."
Description =
"Download assets to this directory. " +
"If not specified, the asset directory path will be derived from the output path."
)]
public string? AssetsDirPath
{
@@ -120,7 +126,10 @@ public abstract class ExportCommandBase : DiscordCommandBase
[CommandOption(
"fuck-russia",
Description = "Don't print the Support Ukraine message to the console."
EnvironmentVariable = "FUCK_RUSSIA",
Description = "Don't print the Support Ukraine message to the console.",
// Use a converter to accept '1' as 'true' to reuse the existing environment variable
Converter = typeof(TruthyBooleanBindingConverter)
)]
public bool IsUkraineSupportMessageDisabled { get; init; }
@@ -129,7 +138,7 @@ public abstract class ExportCommandBase : DiscordCommandBase
protected async ValueTask ExecuteAsync(IConsole console, IReadOnlyList<Channel> channels)
{
// Reuse assets option should only be used when the download assets option is set.
// Asset reuse can only be enabled if the download assets option is set
// https://github.com/Tyrrrz/DiscordChatExporter/issues/425
if (ShouldReuseAssets && !ShouldDownloadAssets)
{
@@ -138,7 +147,7 @@ public abstract class ExportCommandBase : DiscordCommandBase
);
}
// Assets directory should only be specified when the download assets option is set
// Assets directory can only be specified if the download assets option is set
if (!string.IsNullOrWhiteSpace(AssetsDirPath) && !ShouldDownloadAssets)
{
throw new CommandException(
@@ -146,8 +155,8 @@ public abstract class ExportCommandBase : DiscordCommandBase
);
}
// Make sure the user does not try to export all channels into a single file.
// Output path must either be a directory, or contain template tokens.
// Make sure the user does not try to export multiple channels into one file.
// Output path must either be a directory or contain template tokens for this to work.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/799
// https://github.com/Tyrrrz/DiscordChatExporter/issues/917
var isValidOutputPath =
@@ -166,10 +175,10 @@ public abstract class ExportCommandBase : DiscordCommandBase
);
}
// Export
var cancellationToken = console.RegisterCancellationHandler();
var errors = new ConcurrentDictionary<Channel, string>();
// Export
await console.Output.WriteLineAsync($"Exporting {channels.Count} channel(s)...");
await console.CreateProgressTicker().StartAsync(async progressContext =>
{
@@ -222,7 +231,7 @@ public abstract class ExportCommandBase : DiscordCommandBase
);
});
// Print result
// Print the result
using (console.WithForegroundColor(ConsoleColor.White))
{
await console.Output.WriteLineAsync(
@@ -237,28 +246,26 @@ public abstract class ExportCommandBase : DiscordCommandBase
using (console.WithForegroundColor(ConsoleColor.Red))
{
await console.Output.WriteLineAsync(
await console.Error.WriteLineAsync(
$"Failed to export {errors.Count} channel(s):"
);
}
foreach (var (channel, error) in errors)
{
await console.Output.WriteAsync($"{channel.Category.Name} / {channel.Name}: ");
await console.Error.WriteAsync($"{channel.Category.Name} / {channel.Name}: ");
using (console.WithForegroundColor(ConsoleColor.Red))
await console.Output.WriteLineAsync(error);
await console.Error.WriteLineAsync(error);
}
await console.Output.WriteLineAsync();
await console.Error.WriteLineAsync();
}
// Fail the command only if ALL channels failed to export.
// Having some of the channels fail to export is expected.
// If only some channels failed to export, it's okay.
if (errors.Count >= channels.Count)
{
throw new CommandException("Export failed.");
}
}
protected async ValueTask ExecuteAsync(IConsole console, IReadOnlyList<Snowflake> channelIds)
@@ -0,0 +1,26 @@
using System;
using System.Globalization;
using CliFx.Extensibility;
namespace DiscordChatExporter.Cli.Commands.Converters;
internal class TruthyBooleanBindingConverter : BindingConverter<bool>
{
public override bool Convert(string? rawValue)
{
// Null is still considered true, to match the base behavior
if (rawValue is null)
return true;
if (string.IsNullOrWhiteSpace(rawValue))
return false;
if (bool.TryParse(rawValue, out var boolValue))
return boolValue;
if (int.TryParse(rawValue, CultureInfo.InvariantCulture, out var intValue) && intValue == 0)
return false;
return true;
}
}
@@ -13,7 +13,7 @@ using JsonExtensions.Reading;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportall", Description = "Export all accessible channels.")]
[Command("exportall", Description = "Exports all accessible channels.")]
public class ExportAllCommand : ExportCommandBase
{
[CommandOption(
@@ -30,7 +30,9 @@ public class ExportAllCommand : ExportCommandBase
[CommandOption(
"data-package",
Description = "Path to the personal data package (ZIP file) requested from Discord. If provided, only channels referenced in the dump will be exported."
Description =
"Path to the personal data package (ZIP file) requested from Discord. " +
"If provided, only channels referenced in the dump will be exported."
)]
public string? DataPackageFilePath { get; init; }
@@ -62,7 +64,7 @@ public class ExportAllCommand : ExportCommandBase
var entry = archive.GetEntry("messages/index.json");
if (entry is null)
throw new CommandException("Cannot find channel index inside the data package.");
throw new CommandException("Could not find channel index inside the data package.");
await using var stream = entry.Open();
using var document = await JsonDocument.ParseAsync(stream, default, cancellationToken);
@@ -85,7 +87,7 @@ public class ExportAllCommand : ExportCommandBase
}
catch (DiscordChatExporterException)
{
await console.Output.WriteLineAsync($"Channel '{channelName}' ({channelId}) is inaccessible.");
await console.Error.WriteLineAsync($"Channel '{channelName}' ({channelId}) is inaccessible.");
}
}
}
@@ -7,14 +7,16 @@ using DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Cli.Commands;
[Command("export", Description = "Export one or multiple channels.")]
[Command("export", Description = "Exports one or multiple channels.")]
public class ExportChannelsCommand : ExportCommandBase
{
// TODO: change this to plural (breaking change)
[CommandOption(
"channel",
'c',
Description = "Channel ID(s). If provided with category IDs, all channels inside those categories will be exported."
Description =
"Channel ID(s). " +
"If provided with category ID(s), all channels inside those categories will be exported."
)]
public required IReadOnlyList<Snowflake> ChannelIds { get; init; }
@@ -7,7 +7,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportdm", Description = "Export all direct message channels.")]
[Command("exportdm", Description = "Exports all direct message channels.")]
public class ExportDirectMessagesCommand : ExportCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
@@ -9,7 +9,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportguild", Description = "Export all channels within specified guild.")]
[Command("exportguild", Description = "Exports all channels within the specified guild.")]
public class ExportGuildCommand : ExportCommandBase
{
[CommandOption(
@@ -20,6 +20,12 @@ public class GetChannelsCommand : DiscordCommandBase
)]
public required Snowflake GuildId { get; init; }
[CommandOption(
"include-threads",
Description = "Include threads in the output."
)]
public bool IncludeThreads { get; init; }
public override async ValueTask ExecuteAsync(IConsole console)
{
var cancellationToken = console.RegisterCancellationHandler();
@@ -30,11 +36,20 @@ public class GetChannelsCommand : DiscordCommandBase
.ThenBy(c => c.Name)
.ToArray();
var channelIdMaxLength = channels
.Select(c => c.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
var threads = IncludeThreads
? (await Discord.GetGuildThreadsAsync(GuildId, cancellationToken)).OrderBy(c => c.Name).ToArray()
: Array.Empty<ChannelThread>();
foreach (var channel in channels)
{
// Channel ID
await console.Output.WriteAsync(
channel.Id.ToString().PadRight(18, ' ')
channel.Id.ToString().PadRight(channelIdMaxLength, ' ')
);
// Separator
@@ -44,6 +59,39 @@ public class GetChannelsCommand : DiscordCommandBase
// Channel category / name
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteLineAsync($"{channel.Category.Name} / {channel.Name}");
var channelThreads = threads.Where(t => t.ParentId == channel.Id).ToArray();
var channelThreadIdMaxLength = channelThreads
.Select(t => t.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
foreach (var thread in channelThreads)
{
// Indent
await console.Output.WriteAsync(" * ");
// Thread ID
await console.Output.WriteAsync(
thread.Id.ToString().PadRight(channelThreadIdMaxLength, ' ')
);
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
await console.Output.WriteAsync(" | ");
// Thread name
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteAsync($"Thread / {thread.Name}");
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
await console.Output.WriteAsync(" | ");
// Thread status
using (console.WithForegroundColor(ConsoleColor.White))
await console.Output.WriteLineAsync(thread.IsActive ? "Active" : "Archived");
}
}
}
}
@@ -9,7 +9,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("dm", Description = "Get the list of direct message channels.")]
[Command("dm", Description = "Gets the list of all direct message channels.")]
public class GetDirectChannelsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
@@ -22,11 +22,16 @@ public class GetDirectChannelsCommand : DiscordCommandBase
.ThenBy(c => c.Name)
.ToArray();
var channelIdMaxLength = channels
.Select(c => c.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
foreach (var channel in channels)
{
// Channel ID
await console.Output.WriteAsync(
channel.Id.ToString().PadRight(18, ' ')
channel.Id.ToString().PadRight(channelIdMaxLength, ' ')
);
// Separator
@@ -9,7 +9,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("guilds", Description = "Get the list of accessible guilds.")]
[Command("guilds", Description = "Gets the list of accessible guilds.")]
public class GetGuildsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
@@ -19,13 +19,19 @@ public class GetGuildsCommand : DiscordCommandBase
var guilds = (await Discord.GetUserGuildsAsync(cancellationToken))
// Show direct messages first
.OrderByDescending(g => g.Id == Guild.DirectMessages.Id)
.ThenBy(g => g.Name);
.ThenBy(g => g.Name)
.ToArray();
var guildIdMaxLength = guilds
.Select(g => g.Id.ToString().Length)
.OrderDescending()
.FirstOrDefault();
foreach (var guild in guilds)
{
// Guild ID
await console.Output.WriteAsync(
guild.Id.ToString().PadRight(18, ' ')
guild.Id.ToString().PadRight(guildIdMaxLength, ' ')
);
// Separator
@@ -6,7 +6,7 @@ using CliFx.Infrastructure;
namespace DiscordChatExporter.Cli.Commands;
[Command("guide", Description = "Explains how to obtain token, guild or channel ID.")]
[Command("guide", Description = "Explains how to obtain the token, guild or channel ID.")]
public class GuideCommand : ICommand
{
public ValueTask ExecuteAsync(IConsole console)
@@ -41,25 +41,13 @@ public class GuideCommand : ICommand
// Guild or channel ID
using (console.WithForegroundColor(ConsoleColor.White))
console.Output.WriteLine("To get guild ID or guild channel ID:");
console.Output.WriteLine("To get guild ID or channel ID:");
console.Output.WriteLine(" 1. Open Discord");
console.Output.WriteLine(" 2. Open Settings");
console.Output.WriteLine(" 3. Go to Advanced section");
console.Output.WriteLine(" 4. Enable Developer Mode");
console.Output.WriteLine(" 5. Right click on the desired guild or channel and click Copy ID");
console.Output.WriteLine();
// Direct message channel ID
using (console.WithForegroundColor(ConsoleColor.White))
console.Output.WriteLine("To get direct message channel ID:");
console.Output.WriteLine(" 1. Open Discord");
console.Output.WriteLine(" 2. Open the desired direct message channel");
console.Output.WriteLine(" 3. Press Ctrl+Shift+I to show developer tools");
console.Output.WriteLine(" 4. Navigate to the Console tab");
console.Output.WriteLine(" 5. Type \"window.location.href\" and press Enter");
console.Output.WriteLine(" 6. Copy the first long sequence of numbers inside the URL");
console.Output.WriteLine(" 5. Right-click on the desired guild or channel and click Copy Server ID or Copy Channel ID");
console.Output.WriteLine();
// Docs link
@@ -6,11 +6,11 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliFx" Version="2.3.1" />
<PackageReference Include="FuckRussia" Version="1.0.1" PrivateAssets="all" />
<PackageReference Include="Spectre.Console" Version="0.46.0" />
<PackageReference Include="Gress" Version="2.0.1" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.4.0" PrivateAssets="all" />
<PackageReference Include="CliFx" Version="2.3.4" />
<PackageReference Include="Deorcify" Version="1.0.2" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.5.1" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="Spectre.Console" Version="0.47.0" />
</ItemGroup>
<ItemGroup>
@@ -23,19 +23,6 @@ public partial record Channel(
public partial record Channel
{
private static ChannelCategory GetFallbackCategory(ChannelKind channelKind) => new(
Snowflake.Zero,
channelKind switch
{
ChannelKind.GuildTextChat => "Text",
ChannelKind.DirectTextChat => "Private",
ChannelKind.DirectGroupTextChat => "Group",
ChannelKind.GuildNews => "News",
_ => "Default"
},
null
);
public static Channel Parse(JsonElement json, ChannelCategory? categoryHint = null, int? positionHint = null)
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
@@ -45,7 +32,7 @@ public partial record Channel
json.GetPropertyOrNull("guild_id")?.GetNonWhiteSpaceStringOrNull()?.Pipe(Snowflake.Parse) ??
Guild.DirectMessages.Id;
var category = categoryHint ?? GetFallbackCategory(kind);
var category = categoryHint ?? ChannelCategory.CreateDefault(kind);
var name =
// Guild channel
@@ -55,7 +42,7 @@ public partial record Channel
json.GetPropertyOrNull("recipients")?
.EnumerateArrayOrNull()?
.Select(User.Parse)
.Select(u => u.Name)
.Select(u => u.DisplayName)
.Pipe(s => string.Join(", ", s)) ??
// Fallback
@@ -7,6 +7,19 @@ namespace DiscordChatExporter.Core.Discord.Data;
public record ChannelCategory(Snowflake Id, string Name, int? Position) : IHasId
{
public static ChannelCategory CreateDefault(ChannelKind channelKind) => new(
Snowflake.Zero,
channelKind switch
{
ChannelKind.GuildTextChat => "Text",
ChannelKind.DirectTextChat => "Private",
ChannelKind.DirectGroupTextChat => "Group",
ChannelKind.GuildNews => "News",
_ => "Default"
},
null
);
public static ChannelCategory Parse(JsonElement json, int? positionHint = null)
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
@@ -1,7 +1,6 @@
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
// Order of enum fields needs to match the order in the docs.
public enum ChannelKind
{
GuildTextChat = 0,
@@ -0,0 +1,46 @@
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/channel#channel-object-example-thread-channel
public record ChannelThread(
Snowflake Id,
ChannelKind Kind,
Snowflake GuildId,
Snowflake ParentId,
string Name,
bool IsActive,
Snowflake? LastMessageId) : IHasId
{
public static ChannelThread Parse(JsonElement json)
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var kind = (ChannelKind)json.GetProperty("type").GetInt32();
var guildId = json.GetProperty("guild_id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var parentId = json.GetProperty("parent_id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var name = json.GetProperty("name").GetNonWhiteSpaceString();
var isActive = !json
.GetPropertyOrNull("thread_metadata")?
.GetPropertyOrNull("archived")?
.GetBooleanOrNull() ?? true;
var lastMessageId = json
.GetPropertyOrNull("last_message_id")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(Snowflake.Parse);
return new ChannelThread(
id,
kind,
guildId,
parentId,
name,
isActive,
lastMessageId
);
}
}
@@ -1,12 +0,0 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Discord.Data.Common;
public class IdBasedEqualityComparer : IEqualityComparer<IHasId>
{
public static IdBasedEqualityComparer Instance { get; } = new();
public bool Equals(IHasId? x, IHasId? y) => x?.Id == y?.Id;
public int GetHashCode(IHasId obj) => obj.Id.GetHashCode();
}
@@ -45,7 +45,7 @@ public partial record Embed
EmbedKind.Rich;
var url = json.GetPropertyOrNull("url")?.GetNonWhiteSpaceStringOrNull();
var timestamp = json.GetPropertyOrNull("timestamp")?.GetDateTimeOffset();
var timestamp = json.GetPropertyOrNull("timestamp")?.GetDateTimeOffsetOrNull();
var color = json
.GetPropertyOrNull("color")?
@@ -70,7 +70,7 @@ public partial record Embed
// with this by merging related embeds at the end of the message parsing process.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/695
var images =
json.GetPropertyOrNull("image")?.Pipe(EmbedImage.Parse).Enumerate().ToArray() ??
json.GetPropertyOrNull("image")?.Pipe(EmbedImage.Parse).ToSingletonEnumerable().ToArray() ??
Array.Empty<EmbedImage>();
var video = json.GetPropertyOrNull("video")?.Pipe(EmbedVideo.Parse);
@@ -1,6 +1,4 @@
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
public partial record YouTubeVideoEmbedProjection(string VideoId)
{
@@ -9,37 +7,6 @@ public partial record YouTubeVideoEmbedProjection(string VideoId)
public partial record YouTubeVideoEmbedProjection
{
// Adapted from YoutubeExplode
// https://github.com/Tyrrrz/YoutubeExplode/blob/bc700b631bd105d0be208a88116347034bdca88b/YoutubeExplode/Videos/VideoId.cs#L40-L62
private static string? TryParseVideoId(string embedUrl)
{
// Regular URL
// https://www.youtube.com/watch?v=yIVRs6YSbOM
var regularMatch = Regex.Match(embedUrl, @"youtube\..+?/watch.*?v=(.*?)(?:&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(regularMatch))
return regularMatch;
// Short URL
// https://youtu.be/yIVRs6YSbOM
var shortMatch = Regex.Match(embedUrl, @"youtu\.be/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(shortMatch))
return shortMatch;
// Embed URL
// https://www.youtube.com/embed/yIVRs6YSbOM
var embedMatch = Regex.Match(embedUrl, @"youtube\..+?/embed/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(embedMatch))
return embedMatch;
// Shorts URL
// https://www.youtube.com/shorts/sKL1vjP0tIo
var shortsMatch = Regex.Match(embedUrl, @"youtube\..+?/shorts/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(shortsMatch))
return shortsMatch;
return null;
}
public static YouTubeVideoEmbedProjection? TryResolve(Embed embed)
{
if (embed.Kind != EmbedKind.Video)
@@ -48,8 +15,8 @@ public partial record YouTubeVideoEmbedProjection
if (string.IsNullOrWhiteSpace(embed.Url))
return null;
var videoId = TryParseVideoId(embed.Url);
if (string.IsNullOrWhiteSpace(videoId))
var videoId = YoutubeExplode.Videos.VideoId.TryParse(embed.Url);
if (videoId is null)
return null;
return new YouTubeVideoEmbedProjection(videoId);
@@ -34,8 +34,7 @@ public partial record Emoji
if (!string.IsNullOrWhiteSpace(name))
return ImageCdn.GetStandardEmojiUrl(name);
// Either ID or name should be set
throw new ApplicationException("Emoji has neither ID nor name set.");
throw new InvalidOperationException("Either the emoji ID or name should be provided.");
}
public static Emoji Parse(JsonElement json)
@@ -11,7 +11,7 @@ namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/guild#guild-member-object
public partial record Member(
User User,
string? Nick,
string? DisplayName,
string? AvatarUrl,
IReadOnlyList<Snowflake> RoleIds) : IHasId
{
@@ -20,10 +20,12 @@ public partial record Member(
public partial record Member
{
public static Member CreateDefault(User user) => new(user, null, null, Array.Empty<Snowflake>());
public static Member Parse(JsonElement json, Snowflake? guildId = null)
{
var user = json.GetProperty("user").Pipe(User.Parse);
var nick = json.GetPropertyOrNull("nick")?.GetNonWhiteSpaceStringOrNull();
var displayName = json.GetPropertyOrNull("nick")?.GetNonWhiteSpaceStringOrNull();
var roleIds = json
.GetPropertyOrNull("roles")?
@@ -41,7 +43,7 @@ public partial record Member
return new Member(
user,
nick,
displayName,
avatarUrl,
roleIds
);
@@ -108,15 +108,16 @@ public partial record Message
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var kind = (MessageKind)json.GetProperty("type").GetInt32();
var flags = (MessageFlags?)json.GetPropertyOrNull("flags")?.GetInt32() ?? MessageFlags.None;
var flags = (MessageFlags?)json.GetPropertyOrNull("flags")?.GetInt32OrNull() ?? MessageFlags.None;
var author = json.GetProperty("author").Pipe(User.Parse);
var timestamp = json.GetProperty("timestamp").GetDateTimeOffset();
var editedTimestamp = json.GetPropertyOrNull("edited_timestamp")?.GetDateTimeOffset();
var editedTimestamp = json.GetPropertyOrNull("edited_timestamp")?.GetDateTimeOffsetOrNull();
var callEndedTimestamp = json
.GetPropertyOrNull("call")?
.GetPropertyOrNull("ended_timestamp")?
.GetDateTimeOffset();
.GetDateTimeOffsetOrNull();
var isPinned = json.GetPropertyOrNull("pinned")?.GetBooleanOrNull() ?? false;
var content = json.GetPropertyOrNull("content")?.GetStringOrNull() ?? "";
@@ -17,5 +17,5 @@ public enum MessageKind
public static class MessageKindExtensions
{
public static bool IsSystemNotification(this MessageKind c) => (int)c is >= 1 and <= 18;
public static bool IsSystemNotification(this MessageKind kind) => (int)kind is >= 1 and <= 18;
}
@@ -17,7 +17,7 @@ public record Role(Snowflake Id, string Name, int Position, Color? Color) : IHas
var color = json
.GetPropertyOrNull("color")?
.GetInt32()
.GetInt32OrNull()?
.Pipe(System.Drawing.Color.FromArgb)
.ResetAlpha()
.NullIf(c => c.ToRgb() <= 0);
+24 -6
View File
@@ -9,13 +9,24 @@ namespace DiscordChatExporter.Core.Discord.Data;
public partial record User(
Snowflake Id,
bool IsBot,
int Discriminator,
// Remove after Discord migrates all accounts to the new system.
// With that, also remove the DiscriminatorFormatted and FullName properties.
// Replace existing calls to FullName with Name (not DisplayName).
int? Discriminator,
string Name,
string DisplayName,
string AvatarUrl) : IHasId
{
public string DiscriminatorFormatted => $"{Discriminator:0000}";
public string DiscriminatorFormatted => Discriminator is not null
? $"{Discriminator:0000}"
: "0000";
public string FullName => $"{Name}#{DiscriminatorFormatted}";
// This effectively represents the user's true identity.
// In the old system, this is formed from the username and discriminator.
// In the new system, the username is already the user's unique identifier.
public string FullName => Discriminator is not null
? $"{Name}#{DiscriminatorFormatted}"
: Name;
}
public partial record User
@@ -24,16 +35,23 @@ public partial record User
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var isBot = json.GetPropertyOrNull("bot")?.GetBooleanOrNull() ?? false;
var discriminator = json.GetProperty("discriminator").GetNonWhiteSpaceString().Pipe(int.Parse);
var discriminator = json
.GetPropertyOrNull("discriminator")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(int.Parse)
.NullIfDefault();
var name = json.GetProperty("username").GetNonNullString();
var displayName = json.GetPropertyOrNull("global_name")?.GetNonWhiteSpaceStringOrNull() ?? name;
var avatarUrl =
json
.GetPropertyOrNull("avatar")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(h => ImageCdn.GetUserAvatarUrl(id, h)) ??
ImageCdn.GetFallbackUserAvatarUrl(discriminator);
ImageCdn.GetFallbackUserAvatarUrl(discriminator ?? 0);
return new User(id, isBot, discriminator, name, avatarUrl);
return new User(id, isBot, discriminator, name, displayName, avatarUrl);
}
}
+108 -27
View File
@@ -144,19 +144,25 @@ public class DiscordClient
CancellationToken cancellationToken = default)
{
using var response = await GetResponseAsync(url, cancellationToken);
return response.IsSuccessStatusCode
? await response.Content.ReadAsJsonAsync(cancellationToken)
: null;
}
public async ValueTask<User?> TryGetUserAsync(
Snowflake userId,
CancellationToken cancellationToken = default)
{
var response = await TryGetJsonResponseAsync($"users/{userId}", cancellationToken);
return response?.Pipe(User.Parse);
}
public async IAsyncEnumerable<Guild> GetUserGuildsAsync(
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
yield return Guild.DirectMessages;
var currentAfter = Snowflake.Zero;
while (true)
{
var url = new UrlBuilder()
@@ -168,8 +174,9 @@ public class DiscordClient
var response = await GetJsonResponseAsync(url, cancellationToken);
var isEmpty = true;
foreach (var guild in response.EnumerateArray().Select(Guild.Parse))
foreach (var guildJson in response.EnumerateArray())
{
var guild = Guild.Parse(guildJson);
yield return guild;
currentAfter = guild.Id;
@@ -206,34 +213,105 @@ public class DiscordClient
{
var response = await GetJsonResponseAsync($"guilds/{guildId}/channels", cancellationToken);
var responseOrdered = response
var channelsJson = response
.EnumerateArray()
.OrderBy(j => j.GetProperty("position").GetInt32())
.ThenBy(j => j.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse))
.ToArray();
var categories = responseOrdered
.Where(j => j.GetProperty("type").GetInt32() == (int) ChannelKind.GuildCategory)
var categories = channelsJson
.Where(j => j.GetProperty("type").GetInt32() == (int)ChannelKind.GuildCategory)
.Select((j, index) => ChannelCategory.Parse(j, index + 1))
.ToDictionary(j => j.Id.ToString(), StringComparer.Ordinal);
// Discord positions are not deterministic, so we need to normalize them
// because the user may refer to the channel position via file name template.
// Discord channel positions are relative, so we need to normalize them
// so that the user may refer to them more easily in file name templates.
var position = 0;
foreach (var channelJson in responseOrdered)
foreach (var channelJson in channelsJson)
{
var parentId = channelJson.GetPropertyOrNull("parent_id")?.GetNonWhiteSpaceStringOrNull();
var parentId = channelJson
.GetPropertyOrNull("parent_id")?
.GetNonWhiteSpaceStringOrNull();
var category = !string.IsNullOrWhiteSpace(parentId)
? categories.GetValueOrDefault(parentId)
: null;
var channel = Channel.Parse(channelJson, category, position);
yield return Channel.Parse(channelJson, category, position);
position++;
}
}
}
yield return channel;
public async IAsyncEnumerable<ChannelThread> GetGuildThreadsAsync(
Snowflake guildId,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
var tokenKind = _resolvedTokenKind ??= await GetTokenKindAsync(cancellationToken);
var channels = await GetGuildChannelsAsync(guildId, cancellationToken);
// User accounts can only fetch threads using the search endpoint
if (tokenKind == TokenKind.User)
{
foreach (var channel in channels)
{
var currentOffset = 0;
while (true)
{
var url = new UrlBuilder()
.SetPath($"channels/{channel.Id}/threads/search")
.SetQueryParameter("offset", currentOffset.ToString())
.Build();
var response = await TryGetJsonResponseAsync(url, cancellationToken);
if (response is null)
break;
foreach (var threadJson in response.Value.GetProperty("threads").EnumerateArray())
{
yield return ChannelThread.Parse(threadJson);
currentOffset++;
}
if (!response.Value.GetProperty("has_more").GetBoolean())
break;
}
}
}
// Bot accounts can only fetch threads using the threads endpoint
else
{
// Active threads
{
var response = await GetJsonResponseAsync($"guilds/{guildId}/threads/active", cancellationToken);
foreach (var threadJson in response.GetProperty("threads").EnumerateArray())
yield return ChannelThread.Parse(threadJson);
}
foreach (var channel in channels)
{
// Public archived threads
{
var response = await GetJsonResponseAsync(
$"channels/{channel.Id}/threads/archived/public",
cancellationToken
);
foreach (var threadJson in response.GetProperty("threads").EnumerateArray())
yield return ChannelThread.Parse(threadJson);
}
// Private archived threads
{
var response = await GetJsonResponseAsync(
$"channels/{channel.Id}/threads/archived/private",
cancellationToken
);
foreach (var threadJson in response.GetProperty("threads").EnumerateArray())
yield return ChannelThread.Parse(threadJson);
}
}
}
}
@@ -246,7 +324,6 @@ public class DiscordClient
yield break;
var response = await GetJsonResponseAsync($"guilds/{guildId}/roles", cancellationToken);
foreach (var roleJson in response.EnumerateArray())
yield return Role.Parse(roleJson);
}
@@ -280,8 +357,8 @@ public class DiscordClient
var response = await GetJsonResponseAsync($"channels/{channelId}", cancellationToken);
return ChannelCategory.Parse(response);
}
// In some cases, the Discord API returns an empty body when requesting channel category.
// Instead, we use an empty channel category as a fallback.
// In some cases, Discord API returns an empty body when requesting a channel.
// Use an empty channel category as fallback for these cases.
catch (DiscordChatExporterException)
{
return new ChannelCategory(channelId, "Unknown Category", 0);
@@ -294,7 +371,10 @@ public class DiscordClient
{
var response = await GetJsonResponseAsync($"channels/{channelId}", cancellationToken);
var parentId = response.GetPropertyOrNull("parent_id")?.GetNonWhiteSpaceStringOrNull()?.Pipe(Snowflake.Parse);
var parentId = response
.GetPropertyOrNull("parent_id")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(Snowflake.Parse);
var category = parentId is not null
? await GetChannelCategoryAsync(parentId.Value, cancellationToken)
@@ -325,18 +405,18 @@ public class DiscordClient
IProgress<Percentage>? progress = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
// Get the last message in the specified range.
// This snapshots the boundaries, which means that messages posted after the export started
// will not appear in the output.
// Additionally, it provides the date of the last message, which is used to calculate progress.
// Get the last message in the specified range, so we can later calculate the
// progress based on the difference between message timestamps.
// This also snapshots the boundaries, which means that messages posted after
// the export started will not appear in the output.
var lastMessage = await TryGetLastMessageAsync(channelId, before, cancellationToken);
if (lastMessage is null || lastMessage.Timestamp < after?.ToDate())
yield break;
// Keep track of first message in range in order to calculate progress
// Keep track of the first message in range in order to calculate the progress
var firstMessage = default(Message);
var currentAfter = after ?? Snowflake.Zero;
var currentAfter = after ?? Snowflake.Zero;
while (true)
{
var url = new UrlBuilder()
@@ -350,7 +430,8 @@ public class DiscordClient
var messages = response
.EnumerateArray()
.Select(Message.Parse)
.Reverse() // reverse because messages appear newest first
// Messages are returned from newest to oldest, so we need to reverse them
.Reverse()
.ToArray();
// Break if there are no messages (can happen if messages are deleted during execution)
@@ -361,11 +442,11 @@ public class DiscordClient
{
firstMessage ??= message;
// Ensure messages are in range (take into account that last message could have been deleted)
// Ensure that the messages are in range
if (message.Timestamp > lastMessage.Timestamp)
yield break;
// Report progress based on the duration of exported messages divided by total
// Report progress based on timestamps
if (progress is not null)
{
var exportedDuration = (message.Timestamp - firstMessage.Timestamp).Duration();
@@ -373,7 +454,7 @@ public class DiscordClient
progress.Report(Percentage.FromFraction(
// Avoid division by zero if all messages have the exact same timestamp
// (which may be the case if there's only one message in the channel)
// (which happens when there's only one message in the channel)
totalDuration > TimeSpan.Zero
? exportedDuration / totalDuration
: 1
@@ -29,15 +29,11 @@ public partial record struct Snowflake
// As number
if (ulong.TryParse(str, NumberStyles.None, formatProvider, out var value))
{
return new Snowflake(value);
}
// As date
if (DateTimeOffset.TryParse(str, formatProvider, DateTimeStyles.None, out var instant))
{
return FromDate(instant);
}
return null;
}
@@ -59,4 +55,8 @@ public partial record struct Snowflake : IComparable<Snowflake>, IComparable
return Value.CompareTo(other.Value);
}
public static bool operator >(Snowflake left, Snowflake right) => left.CompareTo(right) > 0;
public static bool operator <(Snowflake left, Snowflake right) => left.CompareTo(right) < 0;
}
@@ -1,17 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="AsyncKeyedLock" Version="6.1.1" />
<PackageReference Include="Gress" Version="2.0.1" />
<PackageReference Include="AsyncKeyedLock" Version="6.2.1" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Polly" Version="7.2.3" />
<PackageReference Include="RazorBlade" Version="0.4.2" />
<PackageReference Include="RazorBlade" Version="0.4.3" />
<PackageReference Include="Superpower" Version="3.0.0" />
<PackageReference Include="WebMarkupMin.Core" Version="2.13.8" />
<PackageReference Include="WebMarkupMin.Core" Version="2.14.0" />
<PackageReference Include="YoutubeExplode" Version="6.2.15" />
</ItemGroup>
</Project>
@@ -18,13 +18,20 @@ public class ChannelExporter
IProgress<Percentage>? progress = null,
CancellationToken cancellationToken = default)
{
// Check if the channel is empty
if (request.Channel.LastMessageId is null)
throw DiscordChatExporterException.ChannelIsEmpty();
// Check if the 'after' boundary is valid
if (request.After is not null && request.Channel.LastMessageId < request.After)
throw DiscordChatExporterException.ChannelIsEmpty();
// Build context
var context = new ExportContext(_discord, request);
await context.PopulateChannelsAndRolesAsync(cancellationToken);
// Export messages
await using var messageExporter = new MessageExporter(context);
await foreach (var message in _discord.GetMessagesAsync(
request.Channel.Id,
request.After,
@@ -34,7 +41,7 @@ public class ChannelExporter
{
// Resolve members for referenced users
foreach (var user in message.GetReferencedUsers())
await context.PopulateMemberAsync(user.Id, cancellationToken);
await context.PopulateMemberAsync(user, cancellationToken);
// Export the message
if (request.MessageFilter.IsMatch(message))
@@ -89,11 +89,15 @@ internal partial class CsvMessageWriter : MessageWriter
// Message content
if (message.Kind.IsSystemNotification())
{
await _writer.WriteAsync(CsvEncode(message.GetFallbackContent()));
await _writer.WriteAsync(CsvEncode(
message.GetFallbackContent()
));
}
else
{
await _writer.WriteAsync(CsvEncode(await FormatMarkdownAsync(message.Content, cancellationToken)));
await _writer.WriteAsync(CsvEncode(
await FormatMarkdownAsync(message.Content, cancellationToken)
));
}
await _writer.WriteAsync(',');
@@ -75,9 +75,8 @@ internal partial class ExportAssetDownloader
catch
{
// This can apparently fail for some reason.
// Updating the file date is not a critical task, so we'll just ignore exceptions thrown here.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/585
// Updating file dates is not a critical task, so we'll just
// ignore exceptions thrown here.
}
});
@@ -98,7 +97,7 @@ internal partial class ExportAssetDownloader
{
var urlHash = GetUrlHash(url);
// Try to extract file name from URL
// Try to extract the file name from URL
var fileName = Regex.Match(url, @".+/([^?]*)").Groups[1].Value;
// If it's not there, just use the URL hash as the file name
@@ -110,7 +109,7 @@ internal partial class ExportAssetDownloader
var fileExtension = Path.GetExtension(fileName);
// Probably not a file extension, just a dot in a long file name
// https://github.com/Tyrrrz/DiscordChatExporter/issues/708
// https://github.com/Tyrrrz/DiscordChatExporter/pull/812
if (fileExtension.Length > 41)
{
fileNameWithoutExtension = fileName;
@@ -20,6 +20,7 @@ internal class ExportContext
private readonly ExportAssetDownloader _assetDownloader;
public DiscordClient Discord { get; }
public ExportRequest Request { get; }
public ExportContext(DiscordClient discord,
@@ -43,18 +44,38 @@ internal class ExportContext
_roles[role.Id] = role;
}
// Because members are not pulled in bulk, we need to populate them on demand
public async ValueTask PopulateMemberAsync(Snowflake id, CancellationToken cancellationToken = default)
// Because members cannot be pulled in bulk, we need to populate them on demand
private async ValueTask PopulateMemberAsync(
Snowflake id,
User? fallbackUser,
CancellationToken cancellationToken = default)
{
if (_members.ContainsKey(id))
return;
var member = await Discord.TryGetGuildMemberAsync(Request.Guild.Id, id, cancellationToken);
// User may have left the guild since they were mentioned.
// Create a dummy member object based on the user info.
if (member is null)
{
var user = fallbackUser ?? await Discord.TryGetUserAsync(id, cancellationToken);
// User may have been deleted since they were mentioned
if (user is not null)
member = Member.CreateDefault(user);
}
// Store the result even if it's null, to avoid re-fetching non-existing members
_members[id] = member;
}
public async ValueTask PopulateMemberAsync(Snowflake id, CancellationToken cancellationToken = default) =>
await PopulateMemberAsync(id, null, cancellationToken);
public async ValueTask PopulateMemberAsync(User user, CancellationToken cancellationToken = default) =>
await PopulateMemberAsync(user.Id, user, cancellationToken);
public string FormatDate(DateTimeOffset instant) => Request.DateFormat switch
{
"unix" => instant.ToUnixTimeSeconds().ToString(),
@@ -96,7 +117,7 @@ internal class ExportContext
var relativeFilePath = Path.GetRelativePath(Request.OutputDirPath, filePath);
// Prefer relative paths so that the output files can be copied around without breaking references.
// If the assets path is outside of the export directory, use the absolute path instead.
// If the assets path is outside of the export directory, use an absolute path instead.
var optimalFilePath =
relativeFilePath.StartsWith(".." + Path.DirectorySeparatorChar, StringComparison.Ordinal) ||
relativeFilePath.StartsWith(".." + Path.AltDirectorySeparatorChar, StringComparison.Ordinal)
@@ -117,8 +138,8 @@ internal class ExportContext
// https://github.com/Tyrrrz/DiscordChatExporter/issues/372
catch (Exception ex) when (ex is HttpRequestException or OperationCanceledException)
{
// TODO: add logging so we can be more liberal with catching exceptions
// We don't want this to crash the exporting process in case of failure
// We don't want this to crash the exporting process in case of failure.
// TODO: add logging so we can be more liberal with catching exceptions.
return url;
}
}
@@ -11,6 +11,7 @@ internal class FromMessageFilter : MessageFilter
public override bool IsMatch(Message message) =>
string.Equals(_value, message.Author.Name, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, message.Author.DisplayName, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, message.Author.FullName, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, message.Author.Id.ToString(), StringComparison.OrdinalIgnoreCase);
}
@@ -12,6 +12,7 @@ internal class MentionsMessageFilter : MessageFilter
public override bool IsMatch(Message message) => message.MentionedUsers.Any(user =>
string.Equals(_value, user.Name, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, user.DisplayName, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, user.FullName, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, user.Id.ToString(), StringComparison.OrdinalIgnoreCase)
);
@@ -25,59 +25,59 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
_isJumbo = isJumbo;
}
protected override async ValueTask<MarkdownNode> VisitTextAsync(
protected override ValueTask VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default)
{
_buffer.Append(HtmlEncode(text.Text));
return await base.VisitTextAsync(text, cancellationToken);
return default;
}
protected override async ValueTask<MarkdownNode> VisitFormattingAsync(
protected override async ValueTask VisitFormattingAsync(
FormattingNode formatting,
CancellationToken cancellationToken = default)
{
var (openingTag, closingTag) = formatting.Kind switch
{
FormattingKind.Bold => (
// language=HTML
// lang=html
"<strong>",
// language=HTML
// lang=html
"</strong>"
),
FormattingKind.Italic => (
// language=HTML
// lang=html
"<em>",
// language=HTML
// lang=html
"</em>"
),
FormattingKind.Underline => (
// language=HTML
// lang=html
"<u>",
// language=HTML
// lang=html
"</u>"
),
FormattingKind.Strikethrough => (
// language=HTML
// lang=html
"<s>",
// language=HTML
// lang=html
"</s>"
),
FormattingKind.Spoiler => (
// language=HTML
// lang=html
"""<span class="chatlog__markdown-spoiler chatlog__markdown-spoiler--hidden" onclick="showSpoiler(event, this)">""",
// language=HTML
// lang=html
"""</span>"""
),
FormattingKind.Quote => (
// language=HTML
// lang=html
"""<div class="chatlog__markdown-quote"><div class="chatlog__markdown-quote-border"></div><div class="chatlog__markdown-quote-content">""",
// language=HTML
// lang=html
"""</div></div>"""
),
@@ -85,27 +85,76 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
};
_buffer.Append(openingTag);
var result = await base.VisitFormattingAsync(formatting, cancellationToken);
await VisitAsync(formatting.Children, cancellationToken);
_buffer.Append(closingTag);
return result;
}
protected override async ValueTask<MarkdownNode> VisitInlineCodeBlockAsync(
protected override async ValueTask VisitHeadingAsync(
HeadingNode heading,
CancellationToken cancellationToken = default)
{
_buffer.Append(
// lang=html
$"<h{heading.Level}>"
);
await VisitAsync(heading.Children, cancellationToken);
_buffer.Append(
// lang=html
$"</h{heading.Level}>"
);
}
protected override async ValueTask VisitListAsync(
ListNode list,
CancellationToken cancellationToken = default)
{
_buffer.Append(
// lang=html
"<ul>"
);
await VisitAsync(list.Items, cancellationToken);
_buffer.Append(
// lang=html
"</ul>"
);
}
protected override async ValueTask VisitListItemAsync(
ListItemNode listItem,
CancellationToken cancellationToken = default)
{
_buffer.Append(
// lang=html
"<li>"
);
await VisitAsync(listItem.Children, cancellationToken);
_buffer.Append(
// lang=html
"</li>"
);
}
protected override ValueTask VisitInlineCodeBlockAsync(
InlineCodeBlockNode inlineCodeBlock,
CancellationToken cancellationToken = default)
{
_buffer.Append(
// language=HTML
// lang=html
$"""
<code class="chatlog__markdown-pre chatlog__markdown-pre--inline">{HtmlEncode(inlineCodeBlock.Code)}</code>
"""
);
return await base.VisitInlineCodeBlockAsync(inlineCodeBlock, cancellationToken);
return default;
}
protected override async ValueTask<MarkdownNode> VisitMultiLineCodeBlockAsync(
protected override ValueTask VisitMultiLineCodeBlockAsync(
MultiLineCodeBlockNode multiLineCodeBlock,
CancellationToken cancellationToken = default)
{
@@ -114,16 +163,16 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
: "nohighlight";
_buffer.Append(
// language=HTML
// lang=html
$"""
<code class="chatlog__markdown-pre chatlog__markdown-pre--multiline {highlightClass}">{HtmlEncode(multiLineCodeBlock.Code)}</code>
"""
);
return await base.VisitMultiLineCodeBlockAsync(multiLineCodeBlock, cancellationToken);
return default;
}
protected override async ValueTask<MarkdownNode> VisitLinkAsync(
protected override async ValueTask VisitLinkAsync(
LinkNode link,
CancellationToken cancellationToken = default)
{
@@ -135,21 +184,21 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
_buffer.Append(
!string.IsNullOrWhiteSpace(linkedMessageId)
// language=HTML
// lang=html
? $"""<a href="{HtmlEncode(link.Url)}" onclick="scrollToMessage(event, '{linkedMessageId}')">"""
// language=HTML
// lang=html
: $"""<a href="{HtmlEncode(link.Url)}">"""
);
var result = await base.VisitLinkAsync(link, cancellationToken);
await VisitAsync(link.Children, cancellationToken);
// language=HTML
_buffer.Append("</a>");
return result;
_buffer.Append(
// lang=html
"</a>"
);
}
protected override async ValueTask<MarkdownNode> VisitEmojiAsync(
protected override async ValueTask VisitEmojiAsync(
EmojiNode emoji,
CancellationToken cancellationToken = default)
{
@@ -157,7 +206,7 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
var jumboClass = _isJumbo ? "chatlog__emoji--large" : "";
_buffer.Append(
// language=HTML
// lang=html
$"""
<img
loading="lazy"
@@ -167,18 +216,15 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
src="{await _context.ResolveAssetUrlAsync(emojiImageUrl, cancellationToken)}">
"""
);
return await base.VisitEmojiAsync(emoji, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitMentionAsync(
MentionNode mention,
protected override async ValueTask VisitMentionAsync(MentionNode mention,
CancellationToken cancellationToken = default)
{
if (mention.Kind == MentionKind.Everyone)
{
_buffer.Append(
// language=HTML
// lang=html
"""
<span class="chatlog__markdown-mention">@everyone</span>
"""
@@ -187,7 +233,7 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
else if (mention.Kind == MentionKind.Here)
{
_buffer.Append(
// language=HTML
// lang=html
"""
<span class="chatlog__markdown-mention">@here</span>
"""
@@ -203,12 +249,12 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
var member = mention.TargetId?.Pipe(_context.TryGetMember);
var fullName = member?.User.FullName ?? "Unknown";
var nick = member?.Nick ?? member?.User.Name ?? "Unknown";
var displayName = member?.DisplayName ?? member?.User.DisplayName ?? "Unknown";
_buffer.Append(
// language=HTML
// lang=html
$"""
<span class="chatlog__markdown-mention" title="{HtmlEncode(fullName)}">@{HtmlEncode(nick)}</span>
<span class="chatlog__markdown-mention" title="{HtmlEncode(fullName)}">@{HtmlEncode(displayName)}</span>
"""
);
}
@@ -219,7 +265,7 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
var name = channel?.Name ?? "deleted-channel";
_buffer.Append(
// language=HTML
// lang=html
$"""
<span class="chatlog__markdown-mention">{symbol}{HtmlEncode(name)}</span>
"""
@@ -235,20 +281,18 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
? $"""
color: rgb({color.Value.R}, {color.Value.G}, {color.Value.B}); background-color: rgba({color.Value.R}, {color.Value.G}, {color.Value.B}, 0.1);
"""
: "";
: null;
_buffer.Append(
// language=HTML
// lang=html
$"""
<span class="chatlog__markdown-mention" style="{style}">@{HtmlEncode(name)}</span>
"""
);
}
return await base.VisitMentionAsync(mention, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitTimestampAsync(
protected override ValueTask VisitTimestampAsync(
TimestampNode timestamp,
CancellationToken cancellationToken = default)
{
@@ -261,13 +305,13 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
var formattedLong = timestamp.Instant?.ToLocalString("dddd, MMMM d, yyyy h:mm tt") ?? "";
_buffer.Append(
// language=HTML
// lang=html
$"""
<span class="chatlog__markdown-timestamp" title="{HtmlEncode(formattedLong)}">{HtmlEncode(formatted)}</span>
"""
);
return await base.VisitTimestampAsync(timestamp, cancellationToken);
return default;
}
}
@@ -52,12 +52,12 @@ internal class HtmlMessageWriter : MessageWriter
if ((message.Timestamp - lastMessage.Timestamp).Duration().TotalMinutes > 7)
return false;
// Messages must be from the same author
// Messages must be sent by the same author
if (message.Author.Id != lastMessage.Author.Id)
return false;
// If the user changed their name after the last message, their new messages
// cannot join an existing group.
// If the author changed their name after the last message, their new messages
// cannot join the existing group.
if (!string.Equals(message.Author.FullName, lastMessage.Author.FullName, StringComparison.Ordinal))
return false;
}
@@ -44,7 +44,7 @@ internal class JsonMessageWriter : MessageWriter
_writer.WriteString("id", user.Id.ToString());
_writer.WriteString("name", user.Name);
_writer.WriteString("discriminator", user.DiscriminatorFormatted);
_writer.WriteString("nickname", Context.TryGetMember(user.Id)?.Nick ?? user.Name);
_writer.WriteString("nickname", Context.TryGetMember(user.Id)?.DisplayName ?? user.DisplayName);
_writer.WriteString("color", Context.TryGetUserColor(user.Id)?.ToHex());
_writer.WriteBoolean("isBot", user.IsBot);
@@ -39,7 +39,7 @@ internal partial class MessageExporter : IAsyncDisposable
private async ValueTask<MessageWriter> GetWriterAsync(CancellationToken cancellationToken = default)
{
// Ensure partition limit has not been reached
// Ensure that the partition limit has not been reached
if (_writer is not null &&
_context.Request.PartitionLimit.IsReached(_writer.MessagesWritten, _writer.BytesWritten))
{
@@ -74,11 +74,11 @@ internal partial class MessageExporter
{
private static string GetPartitionFilePath(string baseFilePath, int partitionIndex)
{
// First partition, don't change file name
// First partition, don't change the file name
if (partitionIndex <= 0)
return baseFilePath;
// Inject partition index into file name
// Inject partition index into the file name
var fileNameWithoutExt = Path.GetFileNameWithoutExtension(baseFilePath);
var fileExt = Path.GetExtension(baseFilePath);
var fileName = $"{fileNameWithoutExt} [part {partitionIndex + 1}]{fileExt}";
@@ -40,13 +40,13 @@
var authorMember = Context.TryGetMember(message.Author.Id);
var authorColor = Context.TryGetUserColor(message.Author.Id);
var authorNick = message.Author.IsBot
? message.Author.Name
: authorMember?.Nick ?? message.Author.Name;
var authorDisplayName = message.Author.IsBot
? message.Author.DisplayName
: authorMember?.DisplayName ?? message.Author.DisplayName;
<div id="chatlog__message-container-@message.Id" class="chatlog__message-container @(message.IsPinned ? "chatlog__message-container--pinned" : null)" data-message-id="@message.Id">
<div class="chatlog__message">
@{/* System notification */}
@* System notification *@
@if (message.Kind.IsSystemNotification())
{
<div class="chatlog__message-aside">
@@ -70,18 +70,18 @@
</div>
<div class="chatlog__message-primary">
@{/* Author name */}
<span class="chatlog__system-notification-author" style="@(authorColor is not null ? $"color: rgb({authorColor.Value.R}, {authorColor.Value.G}, {authorColor.Value.B})" : null)" title="@message.Author.FullName" data-user-id="@message.Author.Id">@authorNick</span>
@* Author name *@
<span class="chatlog__system-notification-author" style="@(authorColor is not null ? $"color: rgb({authorColor.Value.R}, {authorColor.Value.G}, {authorColor.Value.B})" : null)" title="@message.Author.FullName" data-user-id="@message.Author.Id">@authorDisplayName</span>
@{/* Space out the content */}
@* Space out the content *@
<span> </span>
@{/* System notification content */}
@* System notification content *@
<span class="chatlog__system-notification-content">
@if (message.Kind == MessageKind.RecipientAdd && message.MentionedUsers.Any())
{
<span>added </span>
<a class="chatlog__system-notification-link" title="@message.MentionedUsers.First().FullName">@message.MentionedUsers.First().Name</a>
<a class="chatlog__system-notification-link" title="@message.MentionedUsers.First().FullName">@message.MentionedUsers.First().DisplayName</a>
<span> to the group.</span>
}
else if (message.Kind == MessageKind.RecipientRemove && message.MentionedUsers.Any())
@@ -93,7 +93,7 @@
else
{
<span>removed </span>
<a class="chatlog__system-notification-link" title="@message.MentionedUsers.First().FullName">@message.MentionedUsers.First().Name</a>
<a class="chatlog__system-notification-link" title="@message.MentionedUsers.First().FullName">@message.MentionedUsers.First().DisplayName</a>
<span> from the group.</span>
}
}
@@ -130,7 +130,7 @@
}
</span>
@{/* Timestamp */}
@* Timestamp *@
<span class="chatlog__system-notification-timestamp">
<a href="#chatlog__message-container-@message.Id">@FormatDate(message.Timestamp)</a>
</span>
@@ -168,12 +168,12 @@
{
var referencedUserMember = Context.TryGetMember(message.ReferencedMessage.Author.Id);
var referencedUserColor = Context.TryGetUserColor(message.ReferencedMessage.Author.Id);
var referencedUserNick = message.ReferencedMessage.Author.IsBot
? message.ReferencedMessage.Author.Name
: referencedUserMember?.Nick ?? message.ReferencedMessage.Author.Name;
var referencedUserDisplayName = message.ReferencedMessage.Author.IsBot
? message.ReferencedMessage.Author.DisplayName
: referencedUserMember?.DisplayName ?? message.ReferencedMessage.Author.DisplayName;
<img class="chatlog__reply-avatar" src="@await ResolveAssetUrlAsync(referencedUserMember?.AvatarUrl ?? message.ReferencedMessage.Author.AvatarUrl)" alt="Avatar" loading="lazy">
<div class="chatlog__reply-author" style="@(referencedUserColor is not null ? $"color: rgb({referencedUserColor.Value.R}, {referencedUserColor.Value.G}, {referencedUserColor.Value.B})" : null)" title="@message.ReferencedMessage.Author.FullName">@referencedUserNick</div>
<div class="chatlog__reply-author" style="@(referencedUserColor is not null ? $"color: rgb({referencedUserColor.Value.R}, {referencedUserColor.Value.G}, {referencedUserColor.Value.B})" : null)" title="@message.ReferencedMessage.Author.FullName">@referencedUserDisplayName</div>
<div class="chatlog__reply-content">
<span class="chatlog__reply-link" onclick="scrollToMessage(event, '@message.ReferencedMessage.Id')">
@if (!string.IsNullOrWhiteSpace(message.ReferencedMessage.Content) && !message.ReferencedMessage.IsContentHidden())
@@ -201,12 +201,12 @@
{
var interactionUserMember = Context.TryGetMember(message.Interaction.User.Id);
var interactionUserColor = Context.TryGetUserColor(message.Interaction.User.Id);
var interactionUserNick = message.Interaction.User.IsBot
? message.Interaction.User.Name
: interactionUserMember?.Nick ?? message.Interaction.User.Name;
var interactionUserDisplayName = message.Interaction.User.IsBot
? message.Interaction.User.DisplayName
: interactionUserMember?.DisplayName ?? message.Interaction.User.DisplayName;
<img class="chatlog__reply-avatar" src="@await ResolveAssetUrlAsync(interactionUserMember?.AvatarUrl ?? message.Interaction.User.AvatarUrl)" alt="Avatar" loading="lazy">
<div class="chatlog__reply-author" style="@(interactionUserColor is not null ? $"color: rgb({interactionUserColor.Value.R}, {interactionUserColor.Value.G}, {interactionUserColor.Value.B})" : null)" title="@message.Interaction.User.FullName">@interactionUserNick</div>
<div class="chatlog__reply-author" style="@(interactionUserColor is not null ? $"color: rgb({interactionUserColor.Value.R}, {interactionUserColor.Value.G}, {interactionUserColor.Value.B})" : null)" title="@message.Interaction.User.FullName">@interactionUserDisplayName</div>
<div class="chatlog__reply-content">
used /@message.Interaction.Name
</div>
@@ -222,10 +222,10 @@
// Header
<div class="chatlog__header">
@{/* Author name */}
<span class="chatlog__author" style="@(authorColor is not null ? $"color: rgb({authorColor.Value.R}, {authorColor.Value.G}, {authorColor.Value.B})" : null)" title="@message.Author.FullName" data-user-id="@message.Author.Id">@authorNick</span>
@* Author name *@
<span class="chatlog__author" style="@(authorColor is not null ? $"color: rgb({authorColor.Value.R}, {authorColor.Value.G}, {authorColor.Value.B})" : null)" title="@message.Author.FullName" data-user-id="@message.Author.Id">@authorDisplayName</span>
@{/* Bot tag */}
@* Bot tag *@
@if (message.Author.IsBot)
{
// For cross-posts, the BOT tag is replaced with the SERVER tag
@@ -239,22 +239,22 @@
}
}
@{/* Timestamp */}
@* Timestamp *@
<span class="chatlog__timestamp"><a href="#chatlog__message-container-@message.Id">@FormatDate(message.Timestamp)</a></span>
</div>
}
@{/* Content */}
@* Content *@
@if ((!string.IsNullOrWhiteSpace(message.Content) && !message.IsContentHidden()) || message.EditedTimestamp is not null)
{
<div class="chatlog__content chatlog__markdown">
@{/* Text */}
@* Text *@
@if (!string.IsNullOrWhiteSpace(message.Content) && !message.IsContentHidden())
{
<span class="chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatMarkdownAsync(message.Content))<!--/wmm:ignore--></span>
}
@{/* Edited timestamp */}
@* Edited timestamp *@
@if (message.EditedTimestamp is not null)
{
<span class="chatlog__edited-timestamp" title="@FormatDate(message.EditedTimestamp.Value)">(edited)</span>
@@ -262,17 +262,17 @@
</div>
}
@{/* Attachments */}
@* Attachments *@
@foreach (var attachment in message.Attachments)
{
<div class="chatlog__attachment @(attachment.IsSpoiler ? "chatlog__attachment--hidden" : null)" onclick="@(attachment.IsSpoiler ? "showSpoiler(event, this)" : null)">
@{/* Spoiler caption */}
@* Spoiler caption *@
@if (attachment.IsSpoiler)
{
<div class="chatlog__attachment-spoiler-caption">SPOILER</div>
}
@{/* Attachment preview */}
@* Attachment preview *@
@if (attachment.IsImage)
{
<a href="@await ResolveAssetUrlAsync(attachment.Url)">
@@ -310,7 +310,7 @@
</div>
}
@{/* Invites */}
@* Invites *@
@{
var inviteCodes = MarkdownParser
.ExtractLinks(message.Content)
@@ -353,7 +353,7 @@
}
}
@{/* Embeds */}
@* Embeds *@
@foreach (var embed in message.Embeds)
{
// Spotify embed
@@ -369,7 +369,7 @@
else if (embed.TryGetYouTubeVideo() is { } youTubeVideoEmbed)
{
<div class="chatlog__embed">
@{/* Color pill */}
@* Color pill *@
@if (embed.Color is not null)
{
<div class="chatlog__embed-color-pill" style="background-color: rgba(@embed.Color.Value.R, @embed.Color.Value.G, @embed.Color.Value.B, @embed.Color.Value.A)"></div>
@@ -382,7 +382,7 @@
<div class="chatlog__embed-content-container">
<div class="chatlog__embed-content">
<div class="chatlog__embed-text">
@{/* Embed author */}
@* Embed author *@
@if (embed.Author is not null)
{
<div class="chatlog__embed-author-container">
@@ -407,7 +407,7 @@
</div>
}
@{/* Embed title */}
@* Embed title *@
@if (!string.IsNullOrWhiteSpace(embed.Title))
{
<div class="chatlog__embed-title">
@@ -424,7 +424,7 @@
</div>
}
@{/* Video player */}
@* Video player *@
<div class="chatlog__embed-youtube-container">
<iframe class="chatlog__embed-youtube" src="@youTubeVideoEmbed.Url" width="400" height="225"></iframe>
</div>
@@ -447,6 +447,19 @@
</a>
</div>
}
// Generic video embed
else if (embed.Kind == EmbedKind.Video && !string.IsNullOrWhiteSpace(embed.Url))
{
var embedVideoUrl =
embed.Video?.ProxyUrl ?? embed.Video?.Url ??
embed.Url;
<div class="chatlog__embed">
<video class="chatlog__embed-generic-video" width="@embed.Video?.Width" height="@embed.Video?.Height" controls>
<source src="@await ResolveAssetUrlAsync(embedVideoUrl)" alt="Embedded video">
</video>
</div>
}
// Generic gifv embed
else if (embed.Kind == EmbedKind.Gifv && !string.IsNullOrWhiteSpace(embed.Url))
{
@@ -455,8 +468,8 @@
embed.Url;
<div class="chatlog__embed">
<video class="chatlog__embed-generic-gifv" loop width="@embed.Video?.Width" height="@embed.Video?.Height" onmouseover="this.play()" onmouseout="this.pause()">
<source src="@await ResolveAssetUrlAsync(embedVideoUrl)" alt="Embedded video">
<video class="chatlog__embed-generic-gifv" width="@embed.Video?.Width" height="@embed.Video?.Height" loop onmouseover="this.play()" onmouseout="this.pause()">
<source src="@await ResolveAssetUrlAsync(embedVideoUrl)" alt="Embedded gifv">
</video>
</div>
}
@@ -464,7 +477,7 @@
else
{
<div class="chatlog__embed">
@{/* Color pill */}
@* Color pill *@
@if (embed.Color is not null)
{
<div class="chatlog__embed-color-pill" style="background-color: rgba(@embed.Color.Value.R, @embed.Color.Value.G, @embed.Color.Value.B, @embed.Color.Value.A)"></div>
@@ -477,7 +490,7 @@
<div class="chatlog__embed-content-container">
<div class="chatlog__embed-content">
<div class="chatlog__embed-text">
@{/* Embed author */}
@* Embed author *@
@if (embed.Author is not null)
{
<div class="chatlog__embed-author-container">
@@ -502,7 +515,7 @@
</div>
}
@{/* Embed title */}
@* Embed title *@
@if (!string.IsNullOrWhiteSpace(embed.Title))
{
<div class="chatlog__embed-title">
@@ -519,7 +532,7 @@
</div>
}
@{/* Embed description */}
@* Embed description *@
@if (!string.IsNullOrWhiteSpace(embed.Description))
{
<div class="chatlog__embed-description">
@@ -527,7 +540,7 @@
</div>
}
@{/* Embed fields */}
@* Embed fields *@
@if (embed.Fields.Any())
{
<div class="chatlog__embed-fields">
@@ -553,7 +566,7 @@
}
</div>
@{/* Embed content */}
@* Embed content *@
@if (embed.Thumbnail is not null && !string.IsNullOrWhiteSpace(embed.Thumbnail.Url))
{
<div class="chatlog__embed-thumbnail-container">
@@ -564,7 +577,7 @@
}
</div>
@{/* Embed images */}
@* Embed images *@
@if (embed.Images.Any())
{
<div class="chatlog__embed-images @(embed.Images.Count == 1 ? "chatlog__embed-images--single" : null)">
@@ -582,18 +595,18 @@
</div>
}
@{/* Embed footer & icon */}
@* Embed footer & icon *@
@if (embed.Footer is not null || embed.Timestamp is not null)
{
<div class="chatlog__embed-footer">
@{/* Footer icon */}
@* Footer icon *@
@if (!string.IsNullOrWhiteSpace(embed.Footer?.IconUrl))
{
<img class="chatlog__embed-footer-icon" src="@await ResolveAssetUrlAsync(embed.Footer.IconProxyUrl ?? embed.Footer.IconUrl)" alt="Footer icon" loading="lazy">
}
<span class="chatlog__embed-footer-text">
@{/* Footer text */}
@* Footer text *@
@if (!string.IsNullOrWhiteSpace(embed.Footer?.Text))
{
@embed.Footer.Text
@@ -604,7 +617,7 @@
@(" • ")
}
@{/* Embed timestamp */}
@* Embed timestamp *@
@if (embed.Timestamp is not null)
{
@FormatDate(embed.Timestamp.Value)
@@ -617,7 +630,7 @@
}
}
@{/* Stickers */}
@* Stickers *@
@foreach (var sticker in message.Stickers)
{
<div class="chatlog__sticker" title="@sticker.Name">
@@ -632,7 +645,7 @@
</div>
}
@{/* Message reactions */}
@* Message reactions *@
@if (message.Reactions.Any())
{
<div class="chatlog__reactions">
@@ -18,15 +18,15 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
_buffer = buffer;
}
protected override async ValueTask<MarkdownNode> VisitTextAsync(
protected override ValueTask VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default)
{
_buffer.Append(text.Text);
return await base.VisitTextAsync(text, cancellationToken);
return default;
}
protected override async ValueTask<MarkdownNode> VisitEmojiAsync(
protected override ValueTask VisitEmojiAsync(
EmojiNode emoji,
CancellationToken cancellationToken = default)
{
@@ -36,11 +36,10 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
: emoji.Name
);
return await base.VisitEmojiAsync(emoji, cancellationToken);
return default;
}
protected override async ValueTask<MarkdownNode> VisitMentionAsync(
MentionNode mention,
protected override async ValueTask VisitMentionAsync(MentionNode mention,
CancellationToken cancellationToken = default)
{
if (mention.Kind == MentionKind.Everyone)
@@ -60,9 +59,9 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
await _context.PopulateMemberAsync(mention.TargetId.Value, cancellationToken);
var member = mention.TargetId?.Pipe(_context.TryGetMember);
var name = member?.User.Name ?? "Unknown";
var displayName = member?.DisplayName ?? member?.User.DisplayName ?? "Unknown";
_buffer.Append($"@{name}");
_buffer.Append($"@{displayName}");
}
else if (mention.Kind == MentionKind.Channel)
{
@@ -82,11 +81,9 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
_buffer.Append($"@{name}");
}
return await base.VisitMentionAsync(mention, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitTimestampAsync(
protected override ValueTask VisitTimestampAsync(
TimestampNode timestamp,
CancellationToken cancellationToken = default)
{
@@ -98,7 +95,7 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
: "Invalid date"
);
return await base.VisitTimestampAsync(timestamp, cancellationToken);
return default;
}
}
@@ -9,13 +9,13 @@ internal static class PlainTextMessageExtensions
public static string GetFallbackContent(this Message message) => message.Kind switch
{
MessageKind.RecipientAdd => message.MentionedUsers.Any()
? $"Added {message.MentionedUsers.First().Name} to the group."
? $"Added {message.MentionedUsers.First().DisplayName} to the group."
: "Added a recipient.",
MessageKind.RecipientRemove => message.MentionedUsers.Any()
? message.Author.Id == message.MentionedUsers.First().Id
? "Left the group."
: $"Removed {message.MentionedUsers.First().Name} from the group."
: $"Removed {message.MentionedUsers.First().DisplayName} from the group."
: "Removed a recipient.",
MessageKind.Call =>
@@ -38,7 +38,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width">
@{/* Styling */}
@* Styling *@
<style>
@@font-face {
src: url("@await ResolveAssetUrlAsync(GetFontUrl("normal", 400))");
@@ -622,24 +622,6 @@
font-weight: 500;
}
.chatlog__embed-generic-image {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-generic-gifv {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-invite-container {
min-width: 320px;
padding: 0.6rem 0.7rem;
@@ -690,6 +672,33 @@
font-weight: 600;
}
.chatlog__embed-generic-image {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-generic-video {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-generic-gifv {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-spotify {
border: 0;
}
@@ -749,6 +758,36 @@
overflow-wrap: break-word;
}
.chatlog__markdown h1 {
margin: 1rem 0 0.5rem;
color: @Themed("#f2f3f5", "#060607");
font-size: 1.5rem;
line-height: 1;
}
.chatlog__markdown h2 {
margin: 1rem 0 0.5rem;
color: @Themed("#f2f3f5", "#060607");
font-size: 1.25rem;
line-height: 1;
}
.chatlog__markdown h3 {
margin: 1rem 0 0.5rem;
color: @Themed("#f2f3f5", "#060607");
font-size: 1rem;
line-height: 1;
}
.chatlog__markdown h1:first-child, h2:first-child, h3:first-child {
margin-top: 0.5rem;
}
.chatlog__markdown ul, ol {
margin: 0 0 0 1rem;
padding: 0;
}
.chatlog__markdown-preserve {
white-space: pre-wrap;
}
@@ -856,7 +895,7 @@
}
</style>
@{/* Syntax highlighting */}
@* Syntax highlighting *@
<link rel="stylesheet" href="@await ResolveAssetUrlAsync($"https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/styles/solarized-{ThemeName.ToLowerInvariant()}.min.css")">
<script src="@await ResolveAssetUrlAsync("https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/highlight.min.js")"></script>
<script>
@@ -865,7 +904,7 @@
});
</script>
@{/* Lottie animation support */}
@* Lottie animation support *@
<script src="@await ResolveAssetUrlAsync("https://cdnjs.cloudflare.com/ajax/libs/lottie-web/5.8.1/lottie.min.js")"></script>
<script>
document.addEventListener('DOMContentLoaded', () => {
@@ -886,7 +925,7 @@
});
</script>
@{/* Scripts */}
@* Scripts *@
<script>
function scrollToMessage(event, id) {
const element = document.getElementById('chatlog__message-container-' + id);
@@ -923,7 +962,7 @@
}
</script>
@{/* Icons */}
@* Icons *@
<svg style="display: none" xmlns="http://www.w3.org/2000/svg">
<defs>
<symbol id="attachment-icon" viewBox="0 0 720 960">
@@ -992,7 +1031,7 @@
</div>
</div>
@{/* Preamble cuts off at this point */}
@* Preamble cuts off at this point *@
<!--wmm:ignore-->
<div class="chatlog">
<!--/wmm:ignore-->
@@ -0,0 +1,8 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown;
internal record HeadingNode(
int Level,
IReadOnlyList<MarkdownNode> Children
) : MarkdownNode, IContainerNode;
@@ -0,0 +1,5 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown;
internal record ListItemNode(IReadOnlyList<MarkdownNode> Children) : MarkdownNode, IContainerNode;
@@ -0,0 +1,5 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown;
internal record ListNode(IReadOnlyList<ListItemNode> Items) : MarkdownNode;
@@ -24,41 +24,40 @@ internal static partial class MarkdownParser
/* Formatting */
private static readonly IMatcher<MarkdownNode> BoldFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character until the earliest double asterisk not followed by an asterisk
// There must be exactly two closing asterisks.
new Regex(@"\*\*(.+?)\*\*(?!\*)", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Bold, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> ItalicFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character until the earliest single asterisk not preceded or followed by an asterisk
// Opening asterisk must not be followed by whitespace
// Closing asterisk must not be preceded by whitespace
// There must be exactly one closing asterisk.
// Opening asterisk must not be followed by whitespace.
// Closing asterisk must not be preceded by whitespace.
new Regex(@"\*(?!\s)(.+?)(?<!\s|\*)\*(?!\*)", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Italic, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> ItalicBoldFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character until the earliest triple asterisk not followed by an asterisk
// There must be exactly three closing asterisks.
new Regex(@"\*(\*\*.+?\*\*)\*(?!\*)", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Italic, Parse(s.Relocate(m.Groups[1]), BoldFormattingNodeMatcher))
);
private static readonly IMatcher<MarkdownNode> ItalicAltFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character except underscore until an underscore
// Closing underscore must not be followed by a word character
new Regex(@"_([^_]+)_(?!\w)", DefaultRegexOptions | RegexOptions.Singleline),
// Closing underscore must not be followed by a word character.
new Regex(@"_(.+?)_(?!\w)", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Italic, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> UnderlineFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character until the earliest double underscore not followed by an underscore
// There must be exactly two closing underscores.
new Regex(@"__(.+?)__(?!_)", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Underline, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> ItalicUnderlineFormattingNodeMatcher =
new RegexMatcher<MarkdownNode>(
// Capture any character until the earliest triple underscore not followed by an underscore
// There must be exactly three closing underscores.
new Regex(@"_(__.+?__)_(?!_)", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(
FormattingKind.Italic,
@@ -66,62 +65,63 @@ internal static partial class MarkdownParser
)
);
private static readonly IMatcher<MarkdownNode> StrikethroughFormattingNodeMatcher =
new RegexMatcher<MarkdownNode>(
// Capture any character until the earliest double tilde
new Regex(@"~~(.+?)~~", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Strikethrough, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> StrikethroughFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex(@"~~(.+?)~~", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Strikethrough, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> SpoilerFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character until the earliest double pipe
new Regex(@"\|\|(.+?)\|\|", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Spoiler, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> SingleLineQuoteNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character until the end of the line
// Opening 'greater than' character must be followed by whitespace
// Text content is optional
new Regex(@"^>\s(.*\n?)", DefaultRegexOptions),
// Include the linebreak in the content so that the lines are preserved in quotes.
new Regex(@"^>\s(.+\n?)", DefaultRegexOptions),
(s, m) => new FormattingNode(FormattingKind.Quote, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> RepeatedSingleLineQuoteNodeMatcher =
new RegexMatcher<MarkdownNode>(
// Repeatedly capture any character until the end of the line
// This one is tricky as it ends up producing multiple separate captures which need to be joined
new Regex(@"(?:^>\s(.*\n?)){2,}", DefaultRegexOptions),
(_, m) => new FormattingNode(
FormattingKind.Quote,
Parse(
// Combine all captures into a single string
string.Concat(m.Groups[1].Captures.Select(c => c.Value))
)
)
);
private static readonly IMatcher<MarkdownNode> RepeatedSingleLineQuoteNodeMatcher = new RegexMatcher<MarkdownNode>(
// Include the linebreaks in the content, so that the lines are preserved in quotes.
new Regex(@"(?:^>\s(.+\n?)){2,}", DefaultRegexOptions),
(s, m) => new FormattingNode(
FormattingKind.Quote,
m.Groups[1].Captures.SelectMany(c => Parse(s.Relocate(c))).ToArray()
)
);
private static readonly IMatcher<MarkdownNode> MultiLineQuoteNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character until the end of the input
// Opening 'greater than' characters must be followed by whitespace
new Regex(@"^>>>\s(.+)", DefaultRegexOptions | RegexOptions.Singleline),
(s, m) => new FormattingNode(FormattingKind.Quote, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> HeadingNodeMatcher = new RegexMatcher<MarkdownNode>(
// Consume the linebreak so that it's not attached to following nodes.
new Regex(@"^(\#{1,3})\s(.+)\n", DefaultRegexOptions),
(s, m) => new HeadingNode(m.Groups[1].Length, Parse(s.Relocate(m.Groups[2])))
);
private static readonly IMatcher<MarkdownNode> ListNodeMatcher = new RegexMatcher<MarkdownNode>(
// Can be preceded by whitespace, which specifies the list's nesting level.
// Following lines that start with (level+1) whitespace are considered part of the list item.
// Consume the linebreak so that it's not attached to following nodes.
new Regex(@"^(\s*)(?:[\-\*]\s(.+(?:\n\s\1.*)*)?\n?)+", DefaultRegexOptions),
(s, m) => new ListNode(
m.Groups[2].Captures.Select(c => new ListItemNode(Parse(s.Relocate(c)))).ToArray()
)
);
/* Code blocks */
private static readonly IMatcher<MarkdownNode> InlineCodeBlockNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any character except backtick until a backtick
// Blank lines at the beginning and end of content are trimmed
// There can be either one or two backticks, but equal number on both sides
// One or two backticks are allowed, but they must match on both sides.
new Regex(@"(`{1,2})([^`]+)\1", DefaultRegexOptions | RegexOptions.Singleline),
(_, m) => new InlineCodeBlockNode(m.Groups[2].Value.Trim('\r', '\n'))
(_, m) => new InlineCodeBlockNode(m.Groups[2].Value)
);
private static readonly IMatcher<MarkdownNode> MultiLineCodeBlockNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture language identifier and then any character until the earliest triple backtick
// Language identifier is one word immediately after opening backticks, followed immediately by newline
// Blank lines at the beginning and end of content are trimmed
// Language identifier is one word immediately after opening backticks, followed immediately by a linebreak.
// Blank lines at the beginning and at the end of content are trimmed.
new Regex(@"```(?:(\w*)\n)?(.+?)```", DefaultRegexOptions | RegexOptions.Singleline),
(_, m) => new MultiLineCodeBlockNode(m.Groups[1].Value, m.Groups[2].Value.Trim('\r', '\n'))
);
@@ -207,7 +207,7 @@ internal static partial class MarkdownParser
);
private static readonly IMatcher<MarkdownNode> CodedStandardEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture :thinking: for known emoji codes
// Capture :thinking:
new Regex(@":([\w_]+):", DefaultRegexOptions),
(_, m) => EmojiIndex.TryGetName(m.Groups[1].Value)?.Pipe(n => new EmojiNode(n))
);
@@ -224,15 +224,9 @@ internal static partial class MarkdownParser
/* Links */
private static readonly IMatcher<MarkdownNode> TitledLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture [title](link)
new Regex(@"\[(.+?)\]\((.+?)\)", DefaultRegexOptions),
(s, m) => new LinkNode(m.Groups[2].Value, Parse(s.Relocate(m.Groups[1])))
);
private static readonly IMatcher<MarkdownNode> AutoLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any non-whitespace character after http:// or https://
// until the last punctuation character or whitespace
// Any non-whitespace character after http:// or https://
// until the last punctuation character or whitespace.
new Regex(@"(https?://\S*[^\.,:;""'\s])", DefaultRegexOptions),
(_, m) => new LinkNode(m.Groups[1].Value)
);
@@ -243,32 +237,38 @@ internal static partial class MarkdownParser
(_, m) => new LinkNode(m.Groups[1].Value)
);
private static readonly IMatcher<MarkdownNode> MaskedLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture [title](link)
new Regex(@"\[(.+?)\]\((.+?)\)", DefaultRegexOptions),
(s, m) => new LinkNode(m.Groups[2].Value, Parse(s.Relocate(m.Groups[1])))
);
/* Text */
private static readonly IMatcher<MarkdownNode> ShrugTextNodeMatcher = new StringMatcher<MarkdownNode>(
// Capture the shrug kaomoji
// This escapes it from matching for formatting
// Capture the shrug kaomoji.
// This escapes it from matching for formatting.
@"¯\_(ツ)_/¯",
s => new TextNode(s.ToString())
);
private static readonly IMatcher<MarkdownNode> IgnoredEmojiTextNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture some specific emoji that don't get rendered
// This escapes it from matching for emoji
// Capture some specific emoji that don't get rendered.
// This escapes them from matching for emoji.
new Regex(@"([\u26A7\u2640\u2642\u2695\u267E\u00A9\u00AE\u2122])", DefaultRegexOptions),
(_, m) => new TextNode(m.Groups[1].Value)
);
private static readonly IMatcher<MarkdownNode> EscapedSymbolTextNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any "symbol/other" character or surrogate pair preceded by a backslash
// This escapes it from matching for emoji
// Capture any "symbol/other" character or surrogate pair preceded by a backslash.
// This escapes them from matching for emoji.
new Regex(@"\\(\p{So}|\p{Cs}{2})", DefaultRegexOptions),
(_, m) => new TextNode(m.Groups[1].Value)
);
private static readonly IMatcher<MarkdownNode> EscapedCharacterTextNodeMatcher = new RegexMatcher<MarkdownNode>(
// Capture any non-whitespace, non latin alphanumeric character preceded by a backslash
// This escapes it from matching for formatting or other tokens
// Capture any non-whitespace, non latin alphanumeric character preceded by a backslash.
// This escapes them from matching for formatting or other tokens.
new Regex(@"\\([^a-zA-Z0-9\s])", DefaultRegexOptions),
(_, m) => new TextNode(m.Groups[1].Value)
);
@@ -310,9 +310,8 @@ internal static partial class MarkdownParser
}
);
// Combine all matchers into one
// Matchers that have similar patterns are ordered from most specific to least specific
private static readonly IMatcher<MarkdownNode> AggregateNodeMatcher = new AggregateMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> NodeMatcher = new AggregateMatcher<MarkdownNode>(
// Escaped text
ShrugTextNodeMatcher,
IgnoredEmojiTextNodeMatcher,
@@ -331,6 +330,8 @@ internal static partial class MarkdownParser
MultiLineQuoteNodeMatcher,
RepeatedSingleLineQuoteNodeMatcher,
SingleLineQuoteNodeMatcher,
HeadingNodeMatcher,
ListNodeMatcher,
// Code blocks
MultiLineCodeBlockNodeMatcher,
@@ -344,7 +345,7 @@ internal static partial class MarkdownParser
RoleMentionNodeMatcher,
// Links
TitledLinkNodeMatcher,
MaskedLinkNodeMatcher,
AutoLinkNodeMatcher,
HiddenLinkNodeMatcher,
@@ -358,7 +359,7 @@ internal static partial class MarkdownParser
);
// Minimal set of matchers for non-multimedia formats (e.g. plain text)
private static readonly IMatcher<MarkdownNode> MinimalAggregateNodeMatcher = new AggregateMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> MinimalNodeMatcher = new AggregateMatcher<MarkdownNode>(
// Mentions
EveryoneMentionNodeMatcher,
HereMentionNodeMatcher,
@@ -383,13 +384,13 @@ internal static partial class MarkdownParser
internal static partial class MarkdownParser
{
private static IReadOnlyList<MarkdownNode> Parse(StringSegment segment) =>
Parse(segment, AggregateNodeMatcher);
Parse(segment, NodeMatcher);
public static IReadOnlyList<MarkdownNode> Parse(string markdown) =>
Parse(new StringSegment(markdown));
private static IReadOnlyList<MarkdownNode> ParseMinimal(StringSegment segment) =>
Parse(segment, MinimalAggregateNodeMatcher);
Parse(segment, MinimalNodeMatcher);
public static IReadOnlyList<MarkdownNode> ParseMinimal(string markdown) =>
ParseMinimal(new StringSegment(markdown));
@@ -7,82 +7,127 @@ namespace DiscordChatExporter.Core.Markdown.Parsing;
internal abstract class MarkdownVisitor
{
protected virtual ValueTask<MarkdownNode> VisitTextAsync(
protected virtual ValueTask VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default) =>
new(text);
CancellationToken cancellationToken = default) => default;
protected virtual async ValueTask<MarkdownNode> VisitFormattingAsync(
protected virtual async ValueTask VisitFormattingAsync(
FormattingNode formatting,
CancellationToken cancellationToken = default)
{
CancellationToken cancellationToken = default) =>
await VisitAsync(formatting.Children, cancellationToken);
return formatting;
}
protected virtual ValueTask<MarkdownNode> VisitInlineCodeBlockAsync(
protected virtual async ValueTask VisitHeadingAsync(
HeadingNode heading,
CancellationToken cancellationToken = default) =>
await VisitAsync(heading.Children, cancellationToken);
protected virtual async ValueTask VisitListAsync(
ListNode list,
CancellationToken cancellationToken = default) =>
await VisitAsync(list.Items, cancellationToken);
protected virtual async ValueTask VisitListItemAsync(
ListItemNode listItem,
CancellationToken cancellationToken = default) =>
await VisitAsync(listItem.Children, cancellationToken);
protected virtual ValueTask VisitInlineCodeBlockAsync(
InlineCodeBlockNode inlineCodeBlock,
CancellationToken cancellationToken = default) =>
new(inlineCodeBlock);
CancellationToken cancellationToken = default) => default;
protected virtual ValueTask<MarkdownNode> VisitMultiLineCodeBlockAsync(
protected virtual ValueTask VisitMultiLineCodeBlockAsync(
MultiLineCodeBlockNode multiLineCodeBlock,
CancellationToken cancellationToken = default) =>
new(multiLineCodeBlock);
CancellationToken cancellationToken = default) => default;
protected virtual async ValueTask<MarkdownNode> VisitLinkAsync(
protected virtual async ValueTask VisitLinkAsync(
LinkNode link,
CancellationToken cancellationToken = default) =>
await VisitAsync(link.Children, cancellationToken);
protected virtual ValueTask VisitEmojiAsync(
EmojiNode emoji,
CancellationToken cancellationToken = default) => default;
protected virtual ValueTask VisitMentionAsync(
MentionNode mention,
CancellationToken cancellationToken = default) => default;
protected virtual ValueTask VisitTimestampAsync(
TimestampNode timestamp,
CancellationToken cancellationToken = default) => default;
public async ValueTask VisitAsync(
MarkdownNode node,
CancellationToken cancellationToken = default)
{
await VisitAsync(link.Children, cancellationToken);
return link;
}
protected virtual ValueTask<MarkdownNode> VisitEmojiAsync(
EmojiNode emoji,
CancellationToken cancellationToken = default) =>
new(emoji);
protected virtual ValueTask<MarkdownNode> VisitMentionAsync(
MentionNode mention,
CancellationToken cancellationToken = default) =>
new(mention);
protected virtual ValueTask<MarkdownNode> VisitTimestampAsync(
TimestampNode timestamp,
CancellationToken cancellationToken = default) =>
new(timestamp);
public async ValueTask<MarkdownNode> VisitAsync(
MarkdownNode node,
CancellationToken cancellationToken = default) => node switch
if (node is TextNode text)
{
TextNode text =>
await VisitTextAsync(text, cancellationToken),
await VisitTextAsync(text, cancellationToken);
return;
}
FormattingNode formatting =>
await VisitFormattingAsync(formatting, cancellationToken),
if (node is FormattingNode formatting)
{
await VisitFormattingAsync(formatting, cancellationToken);
return;
}
InlineCodeBlockNode inlineCodeBlock =>
await VisitInlineCodeBlockAsync(inlineCodeBlock, cancellationToken),
if (node is HeadingNode heading)
{
await VisitHeadingAsync(heading, cancellationToken);
return;
}
MultiLineCodeBlockNode multiLineCodeBlock =>
await VisitMultiLineCodeBlockAsync(multiLineCodeBlock, cancellationToken),
if (node is ListNode list)
{
await VisitListAsync(list, cancellationToken);
return;
}
LinkNode link =>
await VisitLinkAsync(link, cancellationToken),
if (node is ListItemNode listItem)
{
await VisitListItemAsync(listItem, cancellationToken);
return;
}
EmojiNode emoji =>
await VisitEmojiAsync(emoji, cancellationToken),
if (node is InlineCodeBlockNode inlineCodeBlock)
{
await VisitInlineCodeBlockAsync(inlineCodeBlock, cancellationToken);
return;
}
MentionNode mention =>
await VisitMentionAsync(mention, cancellationToken),
if (node is MultiLineCodeBlockNode multiLineCodeBlock)
{
await VisitMultiLineCodeBlockAsync(multiLineCodeBlock, cancellationToken);
return;
}
TimestampNode timestamp =>
await VisitTimestampAsync(timestamp, cancellationToken),
if (node is LinkNode link)
{
await VisitLinkAsync(link, cancellationToken);
return;
}
_ => throw new ArgumentOutOfRangeException(nameof(node))
};
if (node is EmojiNode emoji)
{
await VisitEmojiAsync(emoji, cancellationToken);
return;
}
if (node is MentionNode mention)
{
await VisitMentionAsync(mention, cancellationToken);
return;
}
if (node is TimestampNode timestamp)
{
await VisitTimestampAsync(timestamp, cancellationToken);
return;
}
throw new ArgumentOutOfRangeException(nameof(node));
}
public async ValueTask VisitAsync(
IEnumerable<MarkdownNode> nodes,
@@ -4,9 +4,9 @@ using System.Threading.Tasks;
namespace DiscordChatExporter.Core.Utils.Extensions;
public static class AsyncExtensions
public static class AsyncCollectionExtensions
{
private static async ValueTask<IReadOnlyList<T>> AggregateAsync<T>(
private static async ValueTask<IReadOnlyList<T>> CollectAsync<T>(
this IAsyncEnumerable<T> asyncEnumerable)
{
var list = new List<T>();
@@ -19,5 +19,5 @@ public static class AsyncExtensions
public static ValueTaskAwaiter<IReadOnlyList<T>> GetAwaiter<T>(
this IAsyncEnumerable<T> asyncEnumerable) =>
asyncEnumerable.AggregateAsync().GetAwaiter();
asyncEnumerable.CollectAsync().GetAwaiter();
}
@@ -4,7 +4,7 @@ namespace DiscordChatExporter.Core.Utils.Extensions;
public static class CollectionExtensions
{
public static IEnumerable<T> Enumerate<T>(this T obj)
public static IEnumerable<T> ToSingletonEnumerable<T>(this T obj)
{
yield return obj;
}
@@ -1,4 +1,5 @@
using System;
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Utils.Extensions;
@@ -10,4 +11,7 @@ public static class GenericExtensions
!predicate(value)
? value
: null;
public static T? NullIfDefault<T>(this T value) where T : struct =>
value.NullIf(v => EqualityComparer<T>.Default.Equals(v, default));
}
+1 -1
View File
@@ -16,7 +16,7 @@ public static class Http
private static bool IsRetryableStatusCode(HttpStatusCode statusCode) =>
statusCode is HttpStatusCode.TooManyRequests or HttpStatusCode.RequestTimeout ||
// Treat all server-side errors as retryable.
// Treat all server-side errors as retryable
// https://github.com/Tyrrrz/DiscordChatExporter/issues/908
(int)statusCode >= 500;
@@ -42,7 +42,7 @@ public class MultiSelectionListBoxBehavior<T> : Behavior<ListBox>
set => SetValue(SelectedItemsProperty, value);
}
// Propagate selected items from model to view
// Propagate selected items from the model to the view
private void SelectItems()
{
_viewHandled = true;
@@ -57,7 +57,7 @@ public class MultiSelectionListBoxBehavior<T> : Behavior<ListBox>
_viewHandled = false;
}
// Propagate selected items from view to model
// Propagate selected items from the view to the model
private void OnListBoxSelectionChanged(object? sender, SelectionChangedEventArgs args)
{
if (_viewHandled) return;
@@ -66,7 +66,6 @@ public class MultiSelectionListBoxBehavior<T> : Behavior<ListBox>
SelectedItems = AssociatedObject.SelectedItems.Cast<T>().ToArray();
}
// Re-select items when the set of items changes
private void OnListBoxItemsChanged(object? sender, NotifyCollectionChangedEventArgs args)
{
if (_viewHandled) return;
@@ -82,7 +81,6 @@ public class MultiSelectionListBoxBehavior<T> : Behavior<ListBox>
((INotifyCollectionChanged) AssociatedObject.Items).CollectionChanged += OnListBoxItemsChanged;
}
/// <inheritdoc />
protected override void OnDetaching()
{
base.OnDetaching();
+2 -5
View File
@@ -17,8 +17,8 @@ public class Bootstrapper : Bootstrapper<RootViewModel>
{
base.OnStart();
// Set default theme
// (preferred theme will be set later, once the settings are loaded)
// Set the default theme.
// Preferred theme will be set later, once the settings are loaded.
App.SetLightTheme();
}
@@ -26,10 +26,7 @@ public class Bootstrapper : Bootstrapper<RootViewModel>
{
base.ConfigureIoC(builder);
// Bind settings as singleton
builder.Bind<SettingsService>().ToSelf().InSingletonScope();
// Bind view model factory
builder.Bind<IViewModelFactory>().ToAbstractFactory();
}
@@ -13,17 +13,17 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="Cogwheel" Version="2.0.1" />
<PackageReference Include="FuckRussia" Version="1.0.1" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.0.1" />
<PackageReference Include="MaterialDesignColors" Version="2.1.1" />
<PackageReference Include="MaterialDesignThemes" Version="4.7.1" />
<PackageReference Include="Cogwheel" Version="2.0.2" />
<PackageReference Include="Deorcify" Version="1.0.2" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.5.1" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="MaterialDesignColors" Version="2.1.4" />
<PackageReference Include="MaterialDesignThemes" Version="4.9.0" />
<PackageReference Include="Microsoft.Xaml.Behaviors.Wpf" Version="1.1.39" />
<PackageReference Include="Ookii.Dialogs.Wpf" Version="5.0.1" />
<PackageReference Include="Onova" Version="2.6.6" />
<PackageReference Include="Stylet" Version="1.3.6" />
<PackageReference Include="Onova" Version="2.6.9" />
<PackageReference Include="PropertyChanged.Fody" Version="4.1.0" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.4.0" PrivateAssets="all" />
<PackageReference Include="Stylet" Version="1.3.6" />
</ItemGroup>
<ItemGroup>
@@ -45,7 +45,7 @@ public partial class SettingsService : SettingsBase
public override void Save()
{
// Clear token if it's not supposed to be persisted
// Clear the token if it's not supposed to be persisted
if (!IsTokenPersisted)
LastToken = null;
+7 -5
View File
@@ -6,13 +6,15 @@ internal static class ProcessEx
{
public static void StartShellExecute(string path)
{
var startInfo = new ProcessStartInfo(path)
using var process = new Process
{
UseShellExecute = true
StartInfo = new ProcessStartInfo
{
FileName = path,
UseShellExecute = true
}
};
using (Process.Start(startInfo))
{
}
process.Start();
}
}
@@ -38,6 +38,8 @@ public class DashboardViewModel : PropertyChangedBase
public string? Token { get; set; }
public bool IsTokenSet => !string.IsNullOrWhiteSpace(Token);
private IReadOnlyDictionary<Guild, IReadOnlyList<Channel>>? ChannelsByGuild { get; set; }
public IReadOnlyList<Guild>? AvailableGuilds => ChannelsByGuild?.Keys.ToArray();
@@ -156,14 +158,11 @@ public class DashboardViewModel : PropertyChangedBase
var exporter = new ChannelExporter(_discord);
var channelProgressPairs = dialog
.Channels!
.Select(c => new
{
Channel = c,
Progress = _progressMuxer.CreateInput()
})
.ToArray();
var channelProgressPairs = dialog.Channels!.Select(c => new
{
Channel = c,
Progress = _progressMuxer.CreateInput()
}).ToArray();
var successfulExportCount = 0;
@@ -213,7 +212,7 @@ public class DashboardViewModel : PropertyChangedBase
}
);
// Notify of overall completion
// Notify of the overall completion
if (successfulExportCount > 0)
{
_eventAggregator.Publish(
@@ -31,9 +31,9 @@ public static class MessageBoxViewModelExtensions
string? cancelButtonText)
{
var viewModel = factory.CreateMessageBoxViewModel();
viewModel.Title = title;
viewModel.Message = message;
viewModel.IsOkButtonVisible = !string.IsNullOrWhiteSpace(okButtonText);
viewModel.OkButtonText = okButtonText;
viewModel.IsCancelButtonVisible = !string.IsNullOrWhiteSpace(cancelButtonText);
@@ -25,7 +25,7 @@ public class DialogManager : IDisposable
void OnDialogOpened(object? openSender, DialogOpenedEventArgs openArgs)
{
void OnScreenClosed(object? closeSender, EventArgs args)
void OnScreenClosed(object? closeSender, EventArgs closeArgs)
{
try
{
@@ -1,8 +1,3 @@
namespace DiscordChatExporter.Gui.ViewModels.Messages;
public class NotificationMessage
{
public string Text { get; }
public NotificationMessage(string text) => Text = text;
}
public record NotificationMessage(string Text);
@@ -62,9 +62,7 @@ public class RootViewModel : Screen, IHandle<NotificationMessage>, IDisposable
_settingsService.Save();
if (await _dialogManager.ShowDialogAsync(dialog) == true)
{
ProcessEx.StartShellExecute("https://tyrrrz.me/ukraine?source=discordchatexporter");
}
}
private async ValueTask CheckForUpdatesAsync()
@@ -106,7 +104,7 @@ public class RootViewModel : Screen, IHandle<NotificationMessage>, IDisposable
_settingsService.Load();
// Sync theme with settings
// Sync the theme with settings
if (_settingsService.IsDarkModeEnabled)
{
App.SetDarkTheme();
@@ -116,7 +114,7 @@ public class RootViewModel : Screen, IHandle<NotificationMessage>, IDisposable
App.SetLightTheme();
}
// App has just been updated, display changelog
// App has just been updated, display the changelog
if (_settingsService.LastAppVersion is not null && _settingsService.LastAppVersion != App.Version)
{
Notifications.Enqueue(
@@ -137,8 +135,7 @@ public class RootViewModel : Screen, IHandle<NotificationMessage>, IDisposable
_updateService.FinalizeUpdate(false);
}
public void Handle(NotificationMessage message) =>
Notifications.Enqueue(message.Text);
public void Handle(NotificationMessage message) => Notifications.Enqueue(message.Text);
public void Dispose() => Notifications.Dispose();
}
@@ -85,7 +85,28 @@
materialDesign:TextFieldAssist.TextBoxViewMargin="0,0,2,0"
BorderThickness="0"
FontSize="16"
Text="{Binding Token, UpdateSourceTrigger=PropertyChanged}" />
Text="{Binding Token, UpdateSourceTrigger=PropertyChanged}">
<TextBox.Style>
<Style BasedOn="{StaticResource {x:Type TextBox}}" TargetType="{x:Type TextBox}">
<Style.Triggers>
<!-- Blur the token when it's out of focus -->
<MultiDataTrigger>
<MultiDataTrigger.Conditions>
<Condition Binding="{Binding IsMouseOver, RelativeSource={RelativeSource AncestorType=materialDesign:Card}}" Value="False" />
<Condition Binding="{Binding IsFocused, RelativeSource={RelativeSource Self}}" Value="False" />
<!-- Don't blur if the token is not set, so the user can see the hint text -->
<Condition Binding="{Binding IsTokenSet}" Value="True" />
</MultiDataTrigger.Conditions>
<Setter Property="Effect">
<Setter.Value>
<BlurEffect Radius="12" />
</Setter.Value>
</Setter>
</MultiDataTrigger>
</Style.Triggers>
</Style>
</TextBox.Style>
</TextBox>
<!-- Pull data button -->
<Button
@@ -221,7 +221,7 @@
materialDesign:HintAssist.IsFloating="True"
Style="{DynamicResource MaterialDesignOutlinedTextBox}"
Text="{Binding PartitionLimitValue}"
ToolTip="Split output into partitions, each limited to this number of messages (e.g. '100') or file size (e.g. '10mb')" />
ToolTip="Split the output into partitions, each limited to the specified number of messages (e.g. '100') or file size (e.g. '10mb')" />
<!-- Filtering -->
<TextBox
+7 -2
View File
@@ -5,16 +5,21 @@
[![Coverage](https://img.shields.io/codecov/c/github/Tyrrrz/DiscordChatExporter/master)](https://codecov.io/gh/Tyrrrz/DiscordChatExporter)
[![Release](https://img.shields.io/github/release/Tyrrrz/DiscordChatExporter.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
[![Downloads](https://img.shields.io/github/downloads/Tyrrrz/DiscordChatExporter/total.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
[![Pulls](https://img.shields.io/docker/pulls/tyrrrz/discordchatexporter)](https://hub.docker.com/r/tyrrrz/discordchatexporter)
[![Discord](https://img.shields.io/discord/869237470565392384?label=discord)](https://discord.gg/2SUWKFnHSm)
[![Donate](https://img.shields.io/badge/donate-$$$-8a2be2.svg)](https://tyrrrz.me/donate)
[![Fuck Russia](https://img.shields.io/badge/fuck-russia-e4181c.svg?labelColor=000000)](https://twitter.com/tyrrrz/status/1495972128977571848)
> 🟡 **Project status**: maintenance mode<sup>[[?]](https://github.com/Tyrrrz/.github/blob/master/docs/project-status.md)</sup>
<p align="center">
<img src="favicon.png" alt="Icon" />
</p>
**DiscordChatExporter** is an application that can be used to export message history from any [Discord](https://discord.com) channel to a file.
It works with direct messages, group messages, and server channels, and supports Discord's dialect of markdown as well as most other rich media features.
> ❔ If you have questions or issues, **please refer to the [docs](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs)**.
> ❔ If you have questions or issues, **please refer to the [docs](.docs)**.
> 💬 If you want to chat, **join my [Discord server](https://discord.gg/2SUWKFnHSm)**.
@@ -112,4 +117,4 @@ The following table lists all available download options:
## Related projects
- [**Chat Analytics**](https://github.com/mlomb/chat-analytics) — solution for analyzing chat patterns of Discord users, using exports produced by **DiscordChatExporter**.
- [**DiscordChatExporter-frontend**](https://github.com/slatinsky/DiscordChatExporter-frontend) — convenient viewer for exports produced by **DiscordChatExporter**.
- [**DiscordChatExporter-frontend**](https://github.com/slatinsky/DiscordChatExporter-frontend) — convenient viewer for exports produced by **DiscordChatExporter**.
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

After

Width:  |  Height:  |  Size: 4.2 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.2 KiB

After

Width:  |  Height:  |  Size: 18 KiB