Compare commits

...

33 Commits

Author SHA1 Message Date
Oleksii Holub c0ea57e9b1 Shorten error messages for when attempting to export a period with no messages (#1201) 2024-02-13 19:13:38 +02:00
Oleksii Holub 7c04db40f8 Skip Twitch clip projections because they can't be rendered locally (#1200) 2024-02-13 18:59:51 +02:00
Oleksii Holub 4588bd0496 Add support for Twitch embed projections (#1199) 2024-02-13 18:48:15 +02:00
Tyrrrz 3a22db14a7 Update GitHub Actions 2024-02-01 21:54:50 +02:00
Oleksii Holub ba72927c37 Fix pathing in docker entrypoint 2024-01-31 21:32:40 +02:00
Tyrrrz 822bab2365 Update action versions 2024-01-31 14:44:28 +02:00
Philippe Vaucher 211db11041 Fix docker user permissions issues (#1194) 2024-01-31 14:39:25 +02:00
Tyrrrz 9b8985a920 Fix tests 2024-01-18 20:39:54 +02:00
Tyrrrz 014712b747 Reorder tests 2024-01-18 20:25:30 +02:00
Oleksii Holub d8e43d89be Add support for has:invite filter that matches messages with guild invites (#1188) 2024-01-18 20:24:25 +02:00
Tyrrrz 4e69ff317b Fix nullability mismatch in WPF converters 2024-01-15 23:59:16 +02:00
Oleksii Holub 057beaacd6 Use null as the default locale, which resolves to the current system default locale (#1185) 2024-01-15 23:58:28 +02:00
Tyrrrz 982ba6a76c Fix tests 2024-01-11 23:28:11 +02:00
Tyrrrz dccd9a2f08 Cleanup 2024-01-11 22:43:13 +02:00
Tyrrrz 182f24846b Use new array syntax 2024-01-11 22:21:53 +02:00
Oleksii Holub 777f92a3f1 Fix docker user permission issues by creating the user manually (#1183) 2024-01-09 21:28:19 +02:00
Tyrrrz d4be307fb1 Specify the user in dockerfile via username rather than ID 2024-01-08 17:13:48 +02:00
Tyrrrz 2bbb585931 Update license 2024-01-01 03:09:52 +02:00
Tyrrrz 73080ecfa6 Specify comparison type in Replace(...) 2023-12-31 06:38:53 +02:00
Tyrrrz 0745d49f44 Use native method for enumerating runes 2023-12-31 06:24:23 +02:00
Tyrrrz c16e14e9b6 Update readme 2023-12-30 17:57:18 +02:00
Tyrrrz 8e36002ae7 Slightly better naming 2023-12-29 00:17:11 +02:00
Oleksii Holub a2c5d2e2dd Warn about redirected streams in the CLI only when running inside Docker (#1176) 2023-12-29 00:14:43 +02:00
Tyrrrz 4e3deb409c Update NuGet packages 2023-12-29 00:08:16 +02:00
Oleksii Holub 512f181be2 Refer to "guilds" as "servers" in docs and UI elements (#1175) 2023-12-29 00:03:28 +02:00
Tyrrrz e04eb890e6 Improve wording 2023-12-28 23:56:19 +02:00
Tyrrrz 1ad42ee48b Fix typo 2023-12-27 19:25:06 +02:00
Tyrrrz b8a67617bc Set a high default assembly version to avoid auto-update prompts 2023-12-27 16:20:05 +02:00
Tyrrrz 062427d143 Make test assertions more relaxed in terms of user-made content URLs 2023-12-26 02:40:50 +02:00
Tyrrrz 5f343a46d1 Standardize exception messages 2023-12-25 18:59:47 +02:00
Tyrrrz b3a1cbf635 Update NuGet packages 2023-12-10 22:38:34 +02:00
Tyrrrz 619fe9ccf7 Refactor using c# 12 features 2023-12-10 22:38:34 +02:00
LostXOR 174b92cbb0 Fix GUI link in docs going to CLI page (#1167) 2023-11-30 15:31:31 +02:00
85 changed files with 710 additions and 683 deletions
+11
View File
@@ -61,6 +61,17 @@ For more information, please refer to the [Dockerfile](https://github.com/Tyrrrz
To get your Token and Channel IDs, please refer to [this page](Token-and-IDs.md).
## Unix permissions issues
This image was designed with a user running as uid:gid of 1000:1000.
If your current user has different IDs, and you want to generate files directly editable for your user, you might want to run the container like this:
```console
mkdir data # or chown -R $(id -u):$(id -g) data
docker run -it --rm -v $PWD/data:/out --user $(id -u):$(id -g) tyrrrz/discordchatexporter:stable export -t TOKEN -g CHANNELID
```
## Environment variables
DiscordChatExpoter CLI accepts the `DISCORD_TOKEN` environment variable as a fallback for the `--token` option. You can set this variable either with the `--env` Docker option or with a combination of the `--env-file` Docker option and a `.env` file.
+1 -1
View File
@@ -5,7 +5,7 @@
- [Get .NET Core Runtime](Dotnet.md) (Required for CLI; Installed automatically for GUI; Not required in Docker)
- [Windows](Getting-started.md#gui-or-cli) | [macOS](MacOS.md) | [Linux](Linux.md) | [Docker](Docker.md)
- Getting started:
- [Using the GUI](Using-the-CLI.md)
- [Using the GUI](Using-the-GUI.md)
- [Using the CLI](Using-the-CLI.md)
- [File formats](Getting-started.md#file-formats)
+19 -19
View File
@@ -38,16 +38,16 @@ dotnet DiscordChatExporter.Cli.dll
## CLI commands
| Command | Description |
|-------------------------|-----------------------------------------------------|
| export | Exports a channel |
| exportdm | Exports all direct message channels |
| exportguild | Exports all channels within the specified server |
| exportall | Exports all accessible channels |
| channels | Outputs the list of channels in the given server |
| dm | Outputs the list of direct message channels |
| guilds | Outputs the list of accessible servers |
| guide | Explains how to obtain token, guild, and channel ID |
| Command | Description |
|-------------------------|------------------------------------------------------|
| export | Exports a channel |
| exportdm | Exports all direct message channels |
| exportguild | Exports all channels within the specified server |
| exportall | Exports all accessible channels |
| channels | Outputs the list of channels in the given server |
| dm | Outputs the list of direct message channels |
| guilds | Outputs the list of accessible servers |
| guide | Explains how to obtain token, server, and channel ID |
To use the commands, you'll need a token. For the instructions on how to get a token, please refer to [this page](Token-and-IDs.md), or run `dotnet DiscordChatExporter.Cli.dll guide`.
@@ -110,7 +110,7 @@ dotnet DiscordChatExporter.Cli.dll export -t "mfa.Ifrn" -c 53555 -o "C:\Discord
#### Generating the filename and output directory dynamically
You can use template tokens to generate the output file path based on the guild and channel metadata.
You can use template tokens to generate the output file path based on the server and channel metadata.
```console
dotnet DiscordChatExporter.Cli.dll export -t "mfa.Ifrn" -c 53555 -o "C:\Discord Exports\%G\%T\%C.html"
@@ -122,8 +122,8 @@ path: `C:\Discord Exports\My server\Text channels\my-channel.html`
Here is the full list of supported template tokens:
- `%g` - guild ID
- `%G` - guild name
- `%g` - server ID
- `%G` - server name
- `%t` - category ID
- `%T` - category name
- `%c` - channel ID
@@ -217,9 +217,9 @@ Don't forget to quote (") the date if it has spaces!
More info about .NET date
formats [here](https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-date-and-time-format-strings).
### Export channels from a specific guild (server)
### Export channels from a specific server
To export all channels in a specific guild, use the `exportguild` command and provide the guild ID through the `-g|--guild` option:
To export all channels in a specific server, use the `exportguild` command and provide the server ID through the `-g|--guild` option:
```console
dotnet DiscordChatExporter.Cli.dll exportguild -t "mfa.Ifrn" -g 21814
@@ -261,9 +261,9 @@ To exclude DMs, add the `--include-dm false` option.
dotnet DiscordChatExporter.Cli.dll exportall -t "mfa.Ifrn" --include-dm false
```
### List channels in a guild (server)
### List channels in a server
To list the channels available in a specific guild, use the `channels` command and provide the guild ID through the `-g|--guild` option:
To list the channels available in a specific server, use the `channels` command and provide the server ID through the `-g|--guild` option:
```console
dotnet DiscordChatExporter.Cli.dll channels -t "mfa.Ifrn" -g 21814
@@ -277,9 +277,9 @@ To list all DM channels accessible to the current account, use the `dm` command:
dotnet DiscordChatExporter.Cli.dll dm -t "mfa.Ifrn"
```
### List guilds (servers)
### List servers
To list all guilds accessible by the current account, use the `guilds` command:
To list all servers accessible by the current account, use the `guilds` command:
```console
dotnet DiscordChatExporter.Cli.dll guilds -t "mfa.Ifrn" > C:\path\to\output.txt
+1 -1
View File
@@ -94,7 +94,7 @@ body:
required: true
- label: I have provided a descriptive title for this issue
required: true
- label: I have made sure that that this bug is reproducible on the latest version of the application
- label: I have made sure that this bug is reproducible on the latest version of the application
required: true
- label: I have provided all the information needed to reproduce this bug as efficiently as possible
required: true
+5 -5
View File
@@ -24,10 +24,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # 4.1.0
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: Install Docker Buildx
uses: docker/setup-buildx-action@f95db51fddba0c2d1ec667646a06c2ce06100226 # 3.0.0
uses: docker/setup-buildx-action@f95db51fddba0c2d1ec667646a06c2ce06100226 # v3.0.0
- name: Build image
run: >
@@ -38,7 +38,7 @@ jobs:
--output type=tar,dest=DiscordChatExporter.Cli.Docker.tar
- name: Upload artifacts
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # 3.1.3
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
with:
name: DiscordChatExporter.Cli.Docker
path: DiscordChatExporter.Cli.Docker.tar
@@ -56,10 +56,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # 4.1.0
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: Install Docker Buildx
uses: docker/setup-buildx-action@f95db51fddba0c2d1ec667646a06c2ce06100226 # 3.0.0
uses: docker/setup-buildx-action@f95db51fddba0c2d1ec667646a06c2ce06100226 # v3.0.0
- name: Login to DockerHub
run: >
+9 -9
View File
@@ -26,10 +26,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # 4.1.0
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: Install .NET
uses: actions/setup-dotnet@3447fd6a9f9e57506b15f895c5b76d3b197dc7c2 # 3.2.0
uses: actions/setup-dotnet@4d6c8fcf3c8f7a60068d26b594648e99df24cee3 # v4.0.0
with:
dotnet-version: 8.0.x
@@ -51,10 +51,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # 4.1.0
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: Install .NET
uses: actions/setup-dotnet@3447fd6a9f9e57506b15f895c5b76d3b197dc7c2 # 3.2.0
uses: actions/setup-dotnet@4d6c8fcf3c8f7a60068d26b594648e99df24cee3 # v4.0.0
with:
dotnet-version: 8.0.x
@@ -72,7 +72,7 @@ jobs:
DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
- name: Upload coverage
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # 3.1.4
uses: codecov/codecov-action@f30e4959ba63075080d4f7f90cacc18d9f3fafd7 # v4.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
@@ -92,10 +92,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # 4.1.0
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: Install .NET
uses: actions/setup-dotnet@3447fd6a9f9e57506b15f895c5b76d3b197dc7c2 # 3.2.0
uses: actions/setup-dotnet@4d6c8fcf3c8f7a60068d26b594648e99df24cee3 # v4.0.0
with:
dotnet-version: 8.0.x
@@ -108,7 +108,7 @@ jobs:
--configuration Release
- name: Upload artifacts
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # 3.1.3
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
with:
name: ${{ matrix.app }}
path: ${{ matrix.app }}/bin/publish/
@@ -163,7 +163,7 @@ jobs:
steps:
- name: Download artifacts
uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # 3.0.2
uses: actions/download-artifact@6b208ae046db98c579e8a3aa621ab581ff575935 # v4.1.1
with:
name: ${{ matrix.app }}
path: ${{ matrix.app }}/
+1
View File
@@ -2,6 +2,7 @@
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<Version>999.9.9-dev</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Oleksii Holub</Copyright>
<LangVersion>preview</LangVersion>
@@ -13,7 +13,7 @@
<ItemGroup>
<PackageReference Include="AngleSharp" Version="1.0.7" />
<PackageReference Include="coverlet.collector" Version="6.0.0" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.2" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.7" PrivateAssets="all" />
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="2.3.3" PrivateAssets="all" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
@@ -22,8 +22,8 @@
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="8.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
<PackageReference Include="ReflectionMagic" Version="5.0.0" />
<PackageReference Include="xunit" Version="2.6.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.5.3" PrivateAssets="all" />
<PackageReference Include="xunit" Version="2.6.4" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.5.6" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
@@ -57,7 +57,7 @@ public static class ExportWrapper
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { channelId },
ChannelIds = [channelId],
ExportFormat = format,
OutputPath = filePath,
Locale = "en-US",
@@ -27,7 +27,7 @@ public class DateRangeSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.DateRangeTestCases },
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
After = Snowflake.FromDate(after)
@@ -45,24 +45,20 @@ public class DateRangeSpecs
timestamps
.Should()
.BeEquivalentTo(
new[]
{
[
new DateTimeOffset(2021, 07, 24, 13, 49, 13, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 38, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 39, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 40, TimeSpan.Zero),
new DateTimeOffset(2021, 09, 08, 14, 26, 35, TimeSpan.Zero)
},
],
o =>
{
return o.Using<DateTimeOffset>(
o.Using<DateTimeOffset>(
ctx =>
ctx.Subject
.Should()
ctx.Subject.Should()
.BeCloseTo(ctx.Expectation, TimeSpan.FromSeconds(1))
)
.WhenTypeIs<DateTimeOffset>();
}
.WhenTypeIs<DateTimeOffset>()
);
}
@@ -77,7 +73,7 @@ public class DateRangeSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.DateRangeTestCases },
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
Before = Snowflake.FromDate(before)
@@ -95,22 +91,18 @@ public class DateRangeSpecs
timestamps
.Should()
.BeEquivalentTo(
new[]
{
[
new DateTimeOffset(2021, 07, 19, 13, 34, 18, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 19, 15, 58, 48, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 19, 17, 23, 58, TimeSpan.Zero)
},
],
o =>
{
return o.Using<DateTimeOffset>(
o.Using<DateTimeOffset>(
ctx =>
ctx.Subject
.Should()
ctx.Subject.Should()
.BeCloseTo(ctx.Expectation, TimeSpan.FromSeconds(1))
)
.WhenTypeIs<DateTimeOffset>();
}
.WhenTypeIs<DateTimeOffset>()
);
}
@@ -126,7 +118,7 @@ public class DateRangeSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.DateRangeTestCases },
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
Before = Snowflake.FromDate(before),
@@ -145,23 +137,19 @@ public class DateRangeSpecs
timestamps
.Should()
.BeEquivalentTo(
new[]
{
[
new DateTimeOffset(2021, 07, 24, 13, 49, 13, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 38, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 39, TimeSpan.Zero),
new DateTimeOffset(2021, 07, 24, 14, 52, 40, TimeSpan.Zero)
},
],
o =>
{
return o.Using<DateTimeOffset>(
o.Using<DateTimeOffset>(
ctx =>
ctx.Subject
.Should()
ctx.Subject.Should()
.BeCloseTo(ctx.Expectation, TimeSpan.FromSeconds(1))
)
.WhenTypeIs<DateTimeOffset>();
}
.WhenTypeIs<DateTimeOffset>()
);
}
}
@@ -1,4 +1,5 @@
using System.IO;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Infrastructure;
@@ -25,7 +26,7 @@ public class FilterSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.FilterTestCases },
ChannelIds = [ChannelIds.FilterTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
MessageFilter = MessageFilter.Parse("some text")
@@ -37,7 +38,7 @@ public class FilterSpecs
.EnumerateArray()
.Select(j => j.GetProperty("content").GetString())
.Should()
.ContainSingle("Some random text");
.AllSatisfy(c => c.Contains("Some random text", StringComparison.Ordinal));
}
[Fact]
@@ -50,7 +51,7 @@ public class FilterSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.FilterTestCases },
ChannelIds = [ChannelIds.FilterTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
MessageFilter = MessageFilter.Parse("from:Tyrrrz")
@@ -66,7 +67,7 @@ public class FilterSpecs
}
[Fact]
public async Task I_can_filter_the_export_to_only_include_messages_that_contain_the_specified_content()
public async Task I_can_filter_the_export_to_only_include_messages_that_contain_images()
{
// Arrange
using var file = TempFile.Create();
@@ -75,7 +76,7 @@ public class FilterSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.FilterTestCases },
ChannelIds = [ChannelIds.FilterTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
MessageFilter = MessageFilter.Parse("has:image")
@@ -87,7 +88,7 @@ public class FilterSpecs
.EnumerateArray()
.Select(j => j.GetProperty("content").GetString())
.Should()
.ContainSingle("This has image");
.AllSatisfy(c => c.Contains("This has image", StringComparison.Ordinal));
}
[Fact]
@@ -100,7 +101,7 @@ public class FilterSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.FilterTestCases },
ChannelIds = [ChannelIds.FilterTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
MessageFilter = MessageFilter.Parse("has:pin")
@@ -112,7 +113,32 @@ public class FilterSpecs
.EnumerateArray()
.Select(j => j.GetProperty("content").GetString())
.Should()
.ContainSingle("This is pinned");
.AllSatisfy(c => c.Contains("This is pinned", StringComparison.Ordinal));
}
[Fact]
public async Task I_can_filter_the_export_to_only_include_messages_that_contain_guild_invites()
{
// Arrange
using var file = TempFile.Create();
// Act
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = [ChannelIds.FilterTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
MessageFilter = MessageFilter.Parse("has:invite")
}.ExecuteAsync(new FakeConsole());
// Assert
Json.Parse(await File.ReadAllTextAsync(file.Path))
.GetProperty("messages")
.EnumerateArray()
.Select(j => j.GetProperty("content").GetString())
.Should()
.AllSatisfy(c => c.Contains("This has invite", StringComparison.Ordinal));
}
[Fact]
@@ -125,7 +151,7 @@ public class FilterSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.FilterTestCases },
ChannelIds = [ChannelIds.FilterTestCases],
ExportFormat = ExportFormat.Json,
OutputPath = file.Path,
MessageFilter = MessageFilter.Parse("mentions:Tyrrrz")
@@ -137,6 +163,6 @@ public class FilterSpecs
.EnumerateArray()
.Select(j => j.GetProperty("content").GetString())
.Should()
.ContainSingle("This has mention");
.AllSatisfy(c => c.Contains("This has mention", StringComparison.Ordinal));
}
}
@@ -27,13 +27,7 @@ public class HtmlAttachmentSpecs
.QuerySelectorAll("a")
.Select(e => e.GetAttribute("href"))
.Should()
.Contain(
u =>
u.StartsWith(
"https://cdn.discordapp.com/attachments/885587741654536192/885587844964417596/Test.txt",
StringComparison.Ordinal
)
);
.Contain(u => u.Contains("Test.txt", StringComparison.Ordinal));
}
[Fact]
@@ -52,13 +46,7 @@ public class HtmlAttachmentSpecs
.QuerySelectorAll("img")
.Select(e => e.GetAttribute("src"))
.Should()
.Contain(
u =>
u.StartsWith(
"https://cdn.discordapp.com/attachments/885587741654536192/885654862430359613/bird-thumbnail.png",
StringComparison.Ordinal
)
);
.Contain(u => u.Contains("bird-thumbnail.png", StringComparison.Ordinal));
}
[Fact]
@@ -1,4 +1,5 @@
using System.Linq;
using System;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
@@ -54,7 +55,7 @@ public class HtmlEmbedSpecs
.QuerySelectorAll("img")
.Select(e => e.GetAttribute("src"))
.WhereNotNull()
.Where(s => s.EndsWith("f8w05ja8s4e61.png"))
.Where(s => s.Contains("f8w05ja8s4e61.png", StringComparison.Ordinal))
.Should()
.ContainSingle();
}
@@ -91,8 +92,9 @@ public class HtmlEmbedSpecs
.WhereNotNull()
.Where(
s =>
s.EndsWith(
"i_am_currently_feeling_slight_displeasure_of_what_you_have_just_sent_lqrem.mp4"
s.Contains(
"i_am_currently_feeling_slight_displeasure_of_what_you_have_just_sent_lqrem.mp4",
StringComparison.Ordinal
)
)
.Should()
@@ -113,7 +115,7 @@ public class HtmlEmbedSpecs
.QuerySelectorAll("source")
.Select(e => e.GetAttribute("src"))
.WhereNotNull()
.Where(s => s.EndsWith("tooncasm-test-copy.mp4"))
.Where(s => s.Contains("tooncasm-test-copy.mp4", StringComparison.Ordinal))
.Should()
.ContainSingle();
}
@@ -148,6 +150,26 @@ public class HtmlEmbedSpecs
iframeUrl.Should().StartWith("https://open.spotify.com/embed/track/1LHZMWefF9502NPfArRfvP");
}
[Fact(Skip = "Twitch does not allow embeds from inside local HTML files")]
public async Task I_can_export_a_channel_that_contains_a_message_with_a_Twitch_clip_embed()
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/1196
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.EmbedTestCases,
Snowflake.Parse("1207002986128216074")
);
// Assert
var iframeUrl = message.QuerySelector("iframe")?.GetAttribute("src");
iframeUrl
.Should()
.StartWith(
"https://clips.twitch.tv/embed?clip=SpicyMildCiderThisIsSparta--PQhbllrvej_Ee7v"
);
}
[Fact]
public async Task I_can_export_a_channel_that_contains_a_message_with_a_YouTube_video_embed()
{
@@ -176,15 +198,49 @@ public class HtmlEmbedSpecs
);
// Assert
message
var imageUrls = message
.QuerySelectorAll("img")
.Select(e => e.GetAttribute("src"))
.ToArray();
imageUrls
.Should()
.ContainInOrder(
"https://images-ext-1.discordapp.net/external/-n--xW3EHH_3jlrheVkMXHCM7T86b5Ty4-MzXCT4m1Q/https/pbs.twimg.com/media/FVYIzYPWAAAMBqZ.png",
"https://images-ext-2.discordapp.net/external/z5nEmGeEldV-kswydGLhqUsFHbb5AWHtdvc9XT6N5rE/https/pbs.twimg.com/media/FVYJBWJWAAMNAx2.png",
"https://images-ext-2.discordapp.net/external/gnip03SawMB6uZLagN5sRDpA_1Ap1CcEhMbJfK1z6WQ/https/pbs.twimg.com/media/FVYJHiRX0AANZcz.png",
"https://images-ext-2.discordapp.net/external/jl1v6cCbLaGmiwmKU-ZkXnF4cFsJ39f9A3-oEdqPdZs/https/pbs.twimg.com/media/FVYJNZNXwAAPnVG.png"
.Contain(
u =>
u.EndsWith(
"https/pbs.twimg.com/media/FVYIzYPWAAAMBqZ.png",
StringComparison.Ordinal
)
);
imageUrls
.Should()
.Contain(
u =>
u.EndsWith(
"https/pbs.twimg.com/media/FVYJBWJWAAMNAx2.png",
StringComparison.Ordinal
)
);
imageUrls
.Should()
.Contain(
u =>
u.EndsWith(
"https/pbs.twimg.com/media/FVYJHiRX0AANZcz.png",
StringComparison.Ordinal
)
);
imageUrls
.Should()
.Contain(
u =>
u.EndsWith(
"https/pbs.twimg.com/media/FVYJNZNXwAAPnVG.png",
StringComparison.Ordinal
)
);
message.QuerySelectorAll(".chatlog__embed").Should().ContainSingle();
@@ -202,6 +258,6 @@ public class HtmlEmbedSpecs
);
// Assert
message.Text().Should().Contain("DiscordChatExporter TestServer");
message.Text().Should().Contain("DiscordChatExporter Test Server");
}
}
@@ -26,7 +26,7 @@ public class HtmlGroupingSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.GroupingTestCases },
ChannelIds = [ChannelIds.GroupingTestCases],
ExportFormat = ExportFormat.HtmlDark,
OutputPath = file.Path
}.ExecuteAsync(new FakeConsole());
@@ -24,7 +24,7 @@ public class PartitioningSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.DateRangeTestCases },
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.HtmlDark,
OutputPath = filePath,
PartitionLimit = PartitionLimit.Parse("3")
@@ -45,7 +45,7 @@ public class PartitioningSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.DateRangeTestCases },
ChannelIds = [ChannelIds.DateRangeTestCases],
ExportFormat = ExportFormat.HtmlDark,
OutputPath = filePath,
PartitionLimit = PartitionLimit.Parse("1kb")
@@ -24,7 +24,7 @@ public class SelfContainedSpecs
await new ExportChannelsCommand
{
Token = Secrets.DiscordToken,
ChannelIds = new[] { ChannelIds.SelfContainedTestCases },
ChannelIds = [ChannelIds.SelfContainedTestCases],
ExportFormat = ExportFormat.HtmlDark,
OutputPath = filePath,
ShouldDownloadAssets = true
@@ -5,11 +5,9 @@ using PathEx = System.IO.Path;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempDir : IDisposable
internal partial class TempDir(string path) : IDisposable
{
public string Path { get; }
public TempDir(string path) => Path = path;
public string Path { get; } = path;
public void Dispose()
{
@@ -5,11 +5,9 @@ using PathEx = System.IO.Path;
namespace DiscordChatExporter.Cli.Tests.Utils;
internal partial class TempFile : IDisposable
internal partial class TempFile(string path) : IDisposable
{
public string Path { get; }
public TempFile(string path) => Path = path;
public string Path { get; } = path;
public void Dispose()
{
+7 -6
View File
@@ -40,20 +40,21 @@ LABEL org.opencontainers.image.licenses="MIT"
# Alpine image doesn't come with the ICU libraries pre-installed, so we need to install them manually.
# We need the full ICU data because we allow the user to specify any locale for formatting purposes.
RUN apk add --no-cache icu-libs
RUN apk add --no-cache icu-data-full
RUN apk add --no-cache icu-libs icu-data-full
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
ENV LC_ALL=en_US.UTF-8
ENV LANG=en_US.UTF-8
# Use a non-root user to ensure that the files shared with the host are accessible by the host user.
# We can use the default user provided by the base image for this purpose.
# Use a non-root user to ensure that the files shared with the host are accessible by the host user
# https://github.com/Tyrrrz/DiscordChatExporter/issues/851
USER $APP_UID
# https://github.com/Tyrrrz/DiscordChatExporter/issues/1174
RUN apk add --no-cache su-exec
RUN addgroup -S -g 1000 dce && adduser -S -H -G dce -u 1000 dce
# This directory is exposed to the user for mounting purposes, so it's important that it always
# stays the same for backwards compatibility.
WORKDIR /out
COPY --from=build /tmp/app/DiscordChatExporter.Cli/bin/publish /opt/app
ENTRYPOINT ["/opt/app/DiscordChatExporter.Cli"]
COPY docker-entrypoint.sh /opt/app
ENTRYPOINT ["/opt/app/docker-entrypoint.sh"]
@@ -4,6 +4,7 @@ using CliFx;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Cli.Commands.Base;
@@ -37,26 +38,22 @@ public abstract class DiscordCommandBase : ICommand
{
using (console.WithForegroundColor(ConsoleColor.DarkYellow))
{
console
.Error
.WriteLine(
"Warning: Option --bot is deprecated and should not be used. "
+ "The type of the provided token is now inferred automatically. "
+ "Please update your workflows as this option may be completely removed in a future version."
);
console.Error.WriteLine(
"Warning: The --bot option is deprecated and should not be used. "
+ "The token type is now inferred automatically. "
+ "Please update your workflows as this option may be completely removed in a future version."
);
}
}
#pragma warning restore CS0618
// Note about interactivity
if (console.IsOutputRedirected)
// Note about interactivity for Docker
if (console.IsOutputRedirected && Docker.IsRunningInContainer)
{
console
.Output
.WriteLine(
"Note: Output streams are redirected, rich console interactions are disabled. "
+ "If you are running this command in Docker, consider allocating a pseudo-terminal for better user experience (docker run -it ...)."
);
console.Error.WriteLine(
"Note: Output streams are redirected, rich console interactions are disabled. "
+ "If you are running this command in Docker, consider allocating a pseudo-terminal for better user experience (docker run -it ...)."
);
}
return default;
@@ -1,7 +1,6 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
@@ -30,8 +29,8 @@ public abstract class ExportCommandBase : DiscordCommandBase
"output",
'o',
Description = "Output file or directory path. "
+ "Directory path must end with a slash to avoid ambiguity. "
+ "If a directory is specified, file names will be generated automatically. "
+ "If a directory is specified, file names will be generated automatically based on the channel names and export parameters. "
+ "Directory paths must end with a slash to avoid ambiguity. "
+ "Supports template tokens, see the documentation for more info."
)]
public string OutputPath
@@ -118,8 +117,12 @@ public abstract class ExportCommandBase : DiscordCommandBase
)]
public string DateFormat { get; init; } = "MM/dd/yyyy h:mm tt";
[CommandOption("locale", Description = "Locale to use when formatting dates and numbers.")]
public string Locale { get; init; } = CultureInfo.CurrentCulture.Name;
[CommandOption(
"locale",
Description = "Locale to use when formatting dates and numbers. "
+ "If not specified, the default system locale will be used."
)]
public string? Locale { get; init; }
[CommandOption("utc", Description = "Normalize all timestamps to UTC+0.")]
public bool IsUtcNormalizationEnabled { get; init; } = false;
@@ -244,11 +247,9 @@ public abstract class ExportCommandBase : DiscordCommandBase
// Print the result
using (console.WithForegroundColor(ConsoleColor.White))
{
await console
.Output
.WriteLineAsync(
$"Successfully exported {channels.Count - errorsByChannel.Count} channel(s)."
);
await console.Output.WriteLineAsync(
$"Successfully exported {channels.Count - errorsByChannel.Count} channel(s)."
);
}
// Print errors
@@ -258,11 +259,9 @@ public abstract class ExportCommandBase : DiscordCommandBase
using (console.WithForegroundColor(ConsoleColor.Red))
{
await console
.Error
.WriteLineAsync(
$"Failed to export {errorsByChannel.Count} the following channel(s):"
);
await console.Error.WriteLineAsync(
$"Failed to export {errorsByChannel.Count} the following channel(s):"
);
}
foreach (var (channel, error) in errorsByChannel)
@@ -324,51 +323,33 @@ public abstract class ExportCommandBase : DiscordCommandBase
// Support Ukraine callout
if (!IsUkraineSupportMessageDisabled)
{
console
.Output
.WriteLine(
"┌────────────────────────────────────────────────────────────────────┐"
);
console
.Output
.WriteLine(
"│ Thank you for supporting Ukraine <3 │"
);
console
.Output
.WriteLine(
"│ │"
);
console
.Output
.WriteLine(
"│ As Russia wages a genocidal war against my country, │"
);
console
.Output
.WriteLine(
"│ I'm grateful to everyone who continues to │"
);
console
.Output
.WriteLine(
"│ stand with Ukraine in our fight for freedom. │"
);
console
.Output
.WriteLine(
"│ │"
);
console
.Output
.WriteLine(
"│ Learn more: https://tyrrrz.me/ukraine │"
);
console
.Output
.WriteLine(
"└────────────────────────────────────────────────────────────────────┘"
);
console.Output.WriteLine(
"┌────────────────────────────────────────────────────────────────────┐"
);
console.Output.WriteLine(
"│ Thank you for supporting Ukraine <3 │"
);
console.Output.WriteLine(
"│ │"
);
console.Output.WriteLine(
"│ As Russia wages a genocidal war against my country, │"
);
console.Output.WriteLine(
"│ I'm grateful to everyone who continues to │"
);
console.Output.WriteLine(
"│ stand with Ukraine in our fight for freedom. │"
);
console.Output.WriteLine(
"│ │"
);
console.Output.WriteLine(
"│ Learn more: https://tyrrrz.me/ukraine │"
);
console.Output.WriteLine(
"└────────────────────────────────────────────────────────────────────┘"
);
console.Output.WriteLine("");
}
@@ -21,7 +21,7 @@ public class ExportAllCommand : ExportCommandBase
[CommandOption("include-dm", Description = "Include direct message channels.")]
public bool IncludeDirectChannels { get; init; } = true;
[CommandOption("include-guilds", Description = "Include guild channels.")]
[CommandOption("include-guilds", Description = "Include server channels.")]
public bool IncludeGuildChannels { get; init; } = true;
[CommandOption("include-vc", Description = "Include voice channels.")]
@@ -54,9 +54,9 @@ public class ExportAllCommand : ExportCommandBase
await foreach (var guild in Discord.GetUserGuildsAsync(cancellationToken))
{
// Regular channels
await console
.Output
.WriteLineAsync($"Fetching channels for guild '{guild.Name}'...");
await console.Output.WriteLineAsync(
$"Fetching channels for server '{guild.Name}'..."
);
var fetchedChannelsCount = 0;
await console
@@ -94,9 +94,9 @@ public class ExportAllCommand : ExportCommandBase
// Threads
if (ThreadInclusionMode != ThreadInclusionMode.None)
{
await console
.Output
.WriteLineAsync($"Fetching threads for guild '{guild.Name}'...");
await console.Output.WriteLineAsync(
$"Fetching threads for server '{guild.Name}'..."
);
var fetchedThreadsCount = 0;
await console
@@ -126,9 +126,9 @@ public class ExportAllCommand : ExportCommandBase
}
);
await console
.Output
.WriteLineAsync($"Fetched {fetchedThreadsCount} thread(s).");
await console.Output.WriteLineAsync(
$"Fetched {fetchedThreadsCount} thread(s)."
);
}
}
}
@@ -180,9 +180,9 @@ public class ExportAllCommand : ExportCommandBase
using (console.WithForegroundColor(ConsoleColor.Red))
{
await console
.Error
.WriteLineAsync("Failed to access the following channel(s):");
await console.Error.WriteLineAsync(
"Failed to access the following channel(s):"
);
}
foreach (var dumpChannel in inaccessibleChannels)
@@ -1,5 +1,4 @@
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Infrastructure;
@@ -13,10 +12,10 @@ using Spectre.Console;
namespace DiscordChatExporter.Cli.Commands;
[Command("exportguild", Description = "Exports all channels within the specified guild.")]
[Command("exportguild", Description = "Exports all channels within the specified server.")]
public class ExportGuildCommand : ExportCommandBase
{
[CommandOption("guild", 'g', Description = "Guild ID.")]
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; init; }
[CommandOption("include-vc", Description = "Include voice channels.")]
@@ -12,10 +12,10 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("channels", Description = "Get the list of channels in a guild.")]
[Command("channels", Description = "Get the list of channels in a server.")]
public class GetChannelsCommand : DiscordCommandBase
{
[CommandOption("guild", 'g', Description = "Guild ID.")]
[CommandOption("guild", 'g', Description = "Server ID.")]
public required Snowflake GuildId { get; init; }
[CommandOption("include-vc", Description = "Include voice channels.")]
@@ -64,9 +64,9 @@ public class GetChannelsCommand : DiscordCommandBase
foreach (var channel in channels)
{
// Channel ID
await console
.Output
.WriteAsync(channel.Id.ToString().PadRight(channelIdMaxLength, ' '));
await console.Output.WriteAsync(
channel.Id.ToString().PadRight(channelIdMaxLength, ' ')
);
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
@@ -88,11 +88,9 @@ public class GetChannelsCommand : DiscordCommandBase
await console.Output.WriteAsync(" * ");
// Thread ID
await console
.Output
.WriteAsync(
channelThread.Id.ToString().PadRight(channelThreadIdMaxLength, ' ')
);
await console.Output.WriteAsync(
channelThread.Id.ToString().PadRight(channelThreadIdMaxLength, ' ')
);
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
@@ -108,9 +106,9 @@ public class GetChannelsCommand : DiscordCommandBase
// Thread status
using (console.WithForegroundColor(ConsoleColor.White))
await console
.Output
.WriteLineAsync(channelThread.IsArchived ? "Archived" : "Active");
await console.Output.WriteLineAsync(
channelThread.IsArchived ? "Archived" : "Active"
);
}
}
}
@@ -33,9 +33,9 @@ public class GetDirectChannelsCommand : DiscordCommandBase
foreach (var channel in channels)
{
// Channel ID
await console
.Output
.WriteAsync(channel.Id.ToString().PadRight(channelIdMaxLength, ' '));
await console.Output.WriteAsync(
channel.Id.ToString().PadRight(channelIdMaxLength, ' ')
);
// Separator
using (console.WithForegroundColor(ConsoleColor.DarkGray))
@@ -9,7 +9,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("guilds", Description = "Gets the list of accessible guilds.")]
[Command("guilds", Description = "Gets the list of accessible servers.")]
public class GetGuildsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
@@ -6,20 +6,18 @@ using CliFx.Infrastructure;
namespace DiscordChatExporter.Cli.Commands;
[Command("guide", Description = "Explains how to obtain the token, guild or channel ID.")]
[Command("guide", Description = "Explains how to obtain the token, server or channel ID.")]
public class GuideCommand : ICommand
{
public ValueTask ExecuteAsync(IConsole console)
{
// User token
using (console.WithForegroundColor(ConsoleColor.White))
console.Output.WriteLine("To get user token:");
console.Output.WriteLine("To get the token for your personal account:");
console
.Output
.WriteLine(
" * Automating user accounts is technically against TOS — USE AT YOUR OWN RISK!"
);
console.Output.WriteLine(
" * Automating user accounts is technically against TOS — USE AT YOUR OWN RISK!"
);
console.Output.WriteLine(" 1. Open Discord in your web browser and login");
console.Output.WriteLine(" 2. Open any server or direct message channel");
console.Output.WriteLine(" 3. Press Ctrl+Shift+I to show developer tools");
@@ -34,43 +32,44 @@ public class GuideCommand : ICommand
// Bot token
using (console.WithForegroundColor(ConsoleColor.White))
console.Output.WriteLine("To get bot token:");
console.Output.WriteLine("To get the token for your bot:");
console.Output.WriteLine(" 1. Go to Discord developer portal");
console.Output.WriteLine(" 2. Open your application's settings");
console.Output.WriteLine(" 3. Navigate to the Bot section on the left");
console.Output.WriteLine(" 4. Under Token click Copy");
console
.Output
.WriteLine(
" * Your bot needs to have Message Content Intent enabled to read messages"
);
console.Output.WriteLine(
" * Your bot needs to have the Message Content Intent enabled to read messages"
);
console.Output.WriteLine();
// Guild or channel ID
using (console.WithForegroundColor(ConsoleColor.White))
console.Output.WriteLine("To get guild ID or channel ID:");
console.Output.WriteLine("To get the ID of a server or a channel:");
console.Output.WriteLine(" 1. Open Discord");
console.Output.WriteLine(" 2. Open Settings");
console.Output.WriteLine(" 3. Go to Advanced section");
console.Output.WriteLine(" 4. Enable Developer Mode");
console
.Output
.WriteLine(
" 5. Right-click on the desired guild or channel and click Copy Server ID or Copy Channel ID"
);
console.Output.WriteLine(
" 5. Right-click on the desired server or channel and click Copy Server ID or Copy Channel ID"
);
console.Output.WriteLine();
// Docs link
using (console.WithForegroundColor(ConsoleColor.White))
console
.Output
.WriteLine("If you have questions or issues, please refer to the documentation:");
{
console.Output.WriteLine(
"If you have questions or issues, please refer to the documentation:"
);
}
using (console.WithForegroundColor(ConsoleColor.DarkCyan))
console
.Output
.WriteLine("https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs");
{
console.Output.WriteLine(
"https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs"
);
}
return default;
}
@@ -7,11 +7,11 @@
<ItemGroup>
<PackageReference Include="CliFx" Version="2.3.5" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.2" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.7" PrivateAssets="all" />
<PackageReference Include="Deorcify" Version="1.0.2" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.5.1" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.5.2" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="Spectre.Console" Version="0.47.0" />
<PackageReference Include="Spectre.Console" Version="0.48.0" />
</ItemGroup>
<ItemGroup>
@@ -1,7 +1,6 @@
using System;
using System.Globalization;
using System.Linq;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Core.Discord.Data.Common;
@@ -11,7 +10,7 @@ public static class ImageCdn
// Standard emoji are rendered through Twemoji
public static string GetStandardEmojiUrl(string emojiName)
{
var runes = emojiName.GetRunes().ToArray();
var runes = emojiName.EnumerateRunes().ToArray();
// Variant selector rune is skipped in Twemoji IDs,
// except when the emoji also contains a zero-width joiner.
@@ -31,6 +31,9 @@ public partial record Embed(
public SpotifyTrackEmbedProjection? TryGetSpotifyTrack() =>
SpotifyTrackEmbedProjection.TryResolve(this);
public TwitchClipEmbedProjection? TryGetTwitchClip() =>
TwitchClipEmbedProjection.TryResolve(this);
public YouTubeVideoEmbedProjection? TryGetYouTubeVideo() =>
YouTubeVideoEmbedProjection.TryResolve(this);
}
@@ -0,0 +1,53 @@
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds;
public partial record TwitchClipEmbedProjection(string ClipId)
{
public string Url => $"https://clips.twitch.tv/embed?clip={ClipId}&parent=localhost";
}
public partial record TwitchClipEmbedProjection
{
private static string? TryParseClipId(string embedUrl)
{
// https://clips.twitch.tv/SpookyTenuousPidgeonPanicVis
{
var clipId = Regex
.Match(embedUrl, @"clips\.twitch\.tv/(.*?)(?:\?|&|/|$)")
.Groups[1]
.Value;
if (!string.IsNullOrWhiteSpace(clipId))
return clipId;
}
// https://twitch.tv/clip/SpookyTenuousPidgeonPanicVis
{
var clipId = Regex
.Match(embedUrl, @"twitch\.tv/clip/(.*?)(?:\?|&|/|$)")
.Groups[1]
.Value;
if (!string.IsNullOrWhiteSpace(clipId))
return clipId;
}
return null;
}
public static TwitchClipEmbedProjection? TryResolve(Embed embed)
{
if (embed.Kind != EmbedKind.Video)
return null;
if (string.IsNullOrWhiteSpace(embed.Url))
return null;
var clipId = TryParseClipId(embed.Url);
if (string.IsNullOrWhiteSpace(clipId))
return null;
return new TwitchClipEmbedProjection(clipId);
}
}
@@ -100,8 +100,7 @@ public partial record Message
{
// Concatenate all images into one embed
var images = embed
.Images
.Concat(trailingEmbeds.SelectMany(e => e.Images))
.Images.Concat(trailingEmbeds.SelectMany(e => e.Images))
.ToArray();
normalizedEmbeds.Add(embed with { Images = images });
@@ -18,15 +18,11 @@ using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord;
public class DiscordClient
public class DiscordClient(string token)
{
private readonly string _token;
private readonly Uri _baseUri = new("https://discord.com/api/v10/", UriKind.Absolute);
private TokenKind? _resolvedTokenKind;
public DiscordClient(string token) => _token = token;
private async ValueTask<HttpResponseMessage> GetResponseAsync(
string url,
TokenKind tokenKind,
@@ -40,12 +36,10 @@ public class DiscordClient
// Don't validate because the token can have special characters
// https://github.com/Tyrrrz/DiscordChatExporter/issues/828
request
.Headers
.TryAddWithoutValidation(
"Authorization",
tokenKind == TokenKind.Bot ? $"Bot {_token}" : _token
);
request.Headers.TryAddWithoutValidation(
"Authorization",
tokenKind == TokenKind.Bot ? $"Bot {token}" : token
);
var response = await Http.Client.SendAsync(
request,
@@ -61,13 +55,11 @@ public class DiscordClient
// rate limits and that's just way too much effort.
// https://discord.com/developers/docs/topics/rate-limits
var remainingRequestCount = response
.Headers
.TryGetValue("X-RateLimit-Remaining")
.Headers.TryGetValue("X-RateLimit-Remaining")
?.Pipe(s => int.Parse(s, CultureInfo.InvariantCulture));
var resetAfterDelay = response
.Headers
.TryGetValue("X-RateLimit-Reset-After")
.Headers.TryGetValue("X-RateLimit-Reset-After")
?.Pipe(s => double.Parse(s, CultureInfo.InvariantCulture))
.Pipe(TimeSpan.FromSeconds);
@@ -8,11 +8,9 @@ using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Dump;
public partial class DataDump
public partial class DataDump(IReadOnlyList<DataDumpChannel> channels)
{
public IReadOnlyList<DataDumpChannel> Channels { get; }
public DataDump(IReadOnlyList<DataDumpChannel> channels) => Channels = channels;
public IReadOnlyList<DataDumpChannel> Channels { get; } = channels;
}
public partial class DataDump
@@ -48,7 +46,7 @@ public partial class DataDump
if (entry is null)
{
throw new InvalidOperationException(
"Could not find the channel index inside the data package."
"Failed to locate the channel index inside the data package."
);
}
@@ -1,15 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<PackageReference Include="AsyncKeyedLock" Version="6.2.2" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.2" PrivateAssets="all" />
<PackageReference Include="AsyncKeyedLock" Version="6.2.4" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.7" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Polly" Version="8.2.0" />
<PackageReference Include="RazorBlade" Version="0.4.4" />
<PackageReference Include="RazorBlade" Version="0.5.0" />
<PackageReference Include="Superpower" Version="3.0.0" />
<PackageReference Include="WebMarkupMin.Core" Version="2.14.0" />
<PackageReference Include="YoutubeExplode" Version="6.3.7" />
<PackageReference Include="YoutubeExplode" Version="6.3.10" />
</ItemGroup>
</Project>
@@ -2,17 +2,11 @@
namespace DiscordChatExporter.Core.Exceptions;
public class DiscordChatExporterException : Exception
public class DiscordChatExporterException(
string message,
bool isFatal = false,
Exception? innerException = null
) : Exception(message, innerException)
{
public bool IsFatal { get; }
public DiscordChatExporterException(
string message,
bool isFatal = false,
Exception? innerException = null
)
: base(message, innerException)
{
IsFatal = isFatal;
}
public bool IsFatal { get; } = isFatal;
}
@@ -10,8 +10,6 @@ namespace DiscordChatExporter.Core.Exporting;
public class ChannelExporter(DiscordClient discord)
{
private readonly DiscordClient _discord = discord;
public async ValueTask ExportChannelAsync(
ExportRequest request,
IProgress<Percentage>? progress = null,
@@ -22,8 +20,8 @@ public class ChannelExporter(DiscordClient discord)
if (request.Channel.Kind == ChannelKind.GuildForum)
{
throw new DiscordChatExporterException(
$"Channel '{request.Channel.Name}' (#{request.Channel.Id}) "
+ $"of guild '{request.Guild.Name}' (#{request.Guild.Id}) "
$"Channel '{request.Channel.Name}' "
+ $"of guild '{request.Guild.Name}' "
+ $"is a forum and cannot be exported directly. "
+ "You need to pull its threads and export them individually."
);
@@ -33,8 +31,8 @@ public class ChannelExporter(DiscordClient discord)
if (request.Channel.IsEmpty)
{
throw new DiscordChatExporterException(
$"Channel '{request.Channel.Name}' (#{request.Channel.Id}) "
+ $"of guild '{request.Guild.Name}' (#{request.Guild.Id}) "
$"Channel '{request.Channel.Name}' "
+ $"of guild '{request.Guild.Name}' "
+ $"does not contain any messages."
);
}
@@ -43,8 +41,8 @@ public class ChannelExporter(DiscordClient discord)
if (request.After is not null && !request.Channel.MayHaveMessagesAfter(request.After.Value))
{
throw new DiscordChatExporterException(
$"Channel '{request.Channel.Name}' (#{request.Channel.Id}) "
+ $"of guild '{request.Guild.Name}' (#{request.Guild.Id}) "
$"Channel '{request.Channel.Name}' "
+ $"of guild '{request.Guild.Name}' "
+ $"does not contain any messages within the specified period."
);
}
@@ -56,20 +54,20 @@ public class ChannelExporter(DiscordClient discord)
)
{
throw new DiscordChatExporterException(
$"Channel '{request.Channel.Name}' (#{request.Channel.Id}) "
+ $"of guild '{request.Guild.Name}' (#{request.Guild.Id}) "
$"Channel '{request.Channel.Name}' "
+ $"of guild '{request.Guild.Name}' "
+ $"does not contain any messages within the specified period."
);
}
// Build context
var context = new ExportContext(_discord, request);
var context = new ExportContext(discord, request);
await context.PopulateChannelsAndRolesAsync(cancellationToken);
// Export messages
await using var messageExporter = new MessageExporter(context);
await foreach (
var message in _discord.GetMessagesAsync(
var message in discord.GetMessagesAsync(
request.Channel.Id,
request.After,
request.Before,
@@ -1,4 +1,5 @@
using System.Collections.Generic;
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading;
@@ -122,7 +123,7 @@ internal partial class CsvMessageWriter
{
private static string CsvEncode(string value)
{
value = value.Replace("\"", "\"\"");
value = value.Replace("\"", "\"\"", StringComparison.Ordinal);
return $"\"{value}\"";
}
}
@@ -23,9 +23,6 @@ internal partial class ExportAssetDownloader(string workingDirPath, bool reuse)
o.PoolInitialFill = 1;
});
private readonly string _workingDirPath = workingDirPath;
private readonly bool _reuse = reuse;
// File paths of the previously downloaded assets
private readonly Dictionary<string, string> _previousPathsByUrl = new(StringComparer.Ordinal);
@@ -35,7 +32,7 @@ internal partial class ExportAssetDownloader(string workingDirPath, bool reuse)
)
{
var fileName = GetFileNameFromUrl(url);
var filePath = Path.Combine(_workingDirPath, fileName);
var filePath = Path.Combine(workingDirPath, fileName);
using var _ = await Locker.LockAsync(filePath, cancellationToken);
@@ -43,10 +40,10 @@ internal partial class ExportAssetDownloader(string workingDirPath, bool reuse)
return cachedFilePath;
// Reuse existing files if we're allowed to
if (_reuse && File.Exists(filePath))
if (reuse && File.Exists(filePath))
return _previousPathsByUrl[url] = filePath;
Directory.CreateDirectory(_workingDirPath);
Directory.CreateDirectory(workingDirPath);
await Http.ResiliencePipeline.ExecuteAsync(
async innerCancellationToken =>
@@ -60,9 +57,7 @@ internal partial class ExportAssetDownloader(string workingDirPath, bool reuse)
try
{
var lastModified = response
.Content
.Headers
.TryGetValue("Last-Modified")
.Content.Headers.TryGetValue("Last-Modified")
?.Pipe(
s =>
DateTimeOffset.TryParse(
@@ -8,6 +8,7 @@ using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exporting.Filtering;
using DiscordChatExporter.Core.Exporting.Partitioning;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Core.Exporting;
@@ -39,9 +40,9 @@ public partial class ExportRequest
public bool ShouldReuseAssets { get; }
public string Locale { get; }
public string? Locale { get; }
public CultureInfo CultureInfo { get; }
public CultureInfo? CultureInfo { get; }
public bool IsUtcNormalizationEnabled { get; }
@@ -58,7 +59,7 @@ public partial class ExportRequest
bool shouldFormatMarkdown,
bool shouldDownloadAssets,
bool shouldReuseAssets,
string locale,
string? locale,
bool isUtcNormalizationEnabled
)
{
@@ -83,7 +84,7 @@ public partial class ExportRequest
? FormatPath(assetsDirPath, Guild, Channel, After, Before)
: $"{OutputFilePath}_Files{Path.DirectorySeparatorChar}";
CultureInfo = CultureInfo.GetCultureInfo(Locale);
CultureInfo = Locale?.Pipe(CultureInfo.GetCultureInfo);
}
}
@@ -182,9 +183,10 @@ public partial class ExportRequest
=> before?.ToDate().ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)
?? "",
"%d"
=> DateTimeOffset
.Now
.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture),
=> DateTimeOffset.Now.ToString(
"yyyy-MM-dd",
CultureInfo.InvariantCulture
),
"%%" => "%",
_ => m.Value
@@ -3,28 +3,17 @@ using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering;
internal class BinaryExpressionMessageFilter : MessageFilter
internal class BinaryExpressionMessageFilter(
MessageFilter first,
MessageFilter second,
BinaryExpressionKind kind
) : MessageFilter
{
private readonly MessageFilter _first;
private readonly MessageFilter _second;
private readonly BinaryExpressionKind _kind;
public BinaryExpressionMessageFilter(
MessageFilter first,
MessageFilter second,
BinaryExpressionKind kind
)
{
_first = first;
_second = second;
_kind = kind;
}
public override bool IsMatch(Message message) =>
_kind switch
kind switch
{
BinaryExpressionKind.Or => _first.IsMatch(message) || _second.IsMatch(message),
BinaryExpressionKind.And => _first.IsMatch(message) && _second.IsMatch(message),
_ => throw new InvalidOperationException($"Unknown binary expression kind '{_kind}'.")
BinaryExpressionKind.Or => first.IsMatch(message) || second.IsMatch(message),
BinaryExpressionKind.And => first.IsMatch(message) && second.IsMatch(message),
_ => throw new InvalidOperationException($"Unknown binary expression kind '{kind}'.")
};
}
@@ -4,12 +4,8 @@ using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering;
internal class ContainsMessageFilter : MessageFilter
internal class ContainsMessageFilter(string text) : MessageFilter
{
private readonly string _text;
public ContainsMessageFilter(string text) => _text = text;
// Match content within word boundaries, between spaces, or as the whole input.
// For example, "max" shouldn't match on content "our maximum effort",
// but should match on content "our max effort".
@@ -20,20 +16,18 @@ internal class ContainsMessageFilter : MessageFilter
!string.IsNullOrWhiteSpace(content)
&& Regex.IsMatch(
content,
@"(?:\b|\s|^)" + Regex.Escape(_text) + @"(?:\b|\s|$)",
@"(?:\b|\s|^)" + Regex.Escape(text) + @"(?:\b|\s|$)",
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant
);
public override bool IsMatch(Message message) =>
IsMatch(message.Content)
|| message
.Embeds
.Any(
e =>
IsMatch(e.Title)
|| IsMatch(e.Author?.Name)
|| IsMatch(e.Description)
|| IsMatch(e.Footer?.Text)
|| e.Fields.Any(f => IsMatch(f.Name) || IsMatch(f.Value))
);
|| message.Embeds.Any(
e =>
IsMatch(e.Title)
|| IsMatch(e.Author?.Name)
|| IsMatch(e.Description)
|| IsMatch(e.Footer?.Text)
|| e.Fields.Any(f => IsMatch(f.Name) || IsMatch(f.Value))
);
}
@@ -3,15 +3,11 @@ using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering;
internal class FromMessageFilter : MessageFilter
internal class FromMessageFilter(string value) : MessageFilter
{
private readonly string _value;
public FromMessageFilter(string value) => _value = value;
public override bool IsMatch(Message message) =>
string.Equals(_value, message.Author.Name, StringComparison.OrdinalIgnoreCase)
|| string.Equals(_value, message.Author.DisplayName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(_value, message.Author.FullName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(_value, message.Author.Id.ToString(), StringComparison.OrdinalIgnoreCase);
string.Equals(value, message.Author.Name, StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, message.Author.DisplayName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, message.Author.FullName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, message.Author.Id.ToString(), StringComparison.OrdinalIgnoreCase);
}
@@ -1,30 +1,31 @@
using System;
using System.Linq;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Markdown.Parsing;
namespace DiscordChatExporter.Core.Exporting.Filtering;
internal class HasMessageFilter : MessageFilter
internal class HasMessageFilter(MessageContentMatchKind kind) : MessageFilter
{
private readonly MessageContentMatchKind _kind;
public HasMessageFilter(MessageContentMatchKind kind) => _kind = kind;
public override bool IsMatch(Message message) =>
_kind switch
kind switch
{
MessageContentMatchKind.Link
=> Regex.IsMatch(message.Content, "https?://\\S*[^\\.,:;\"\'\\s]"),
MessageContentMatchKind.Link => MarkdownParser.ExtractLinks(message.Content).Any(),
MessageContentMatchKind.Embed => message.Embeds.Any(),
MessageContentMatchKind.File => message.Attachments.Any(),
MessageContentMatchKind.Video => message.Attachments.Any(file => file.IsVideo),
MessageContentMatchKind.Image => message.Attachments.Any(file => file.IsImage),
MessageContentMatchKind.Sound => message.Attachments.Any(file => file.IsAudio),
MessageContentMatchKind.Pin => message.IsPinned,
MessageContentMatchKind.Invite
=> MarkdownParser
.ExtractLinks(message.Content)
.Select(l => l.Url)
.Select(Invite.TryGetCodeFromUrl)
.Any(c => !string.IsNullOrWhiteSpace(c)),
_
=> throw new InvalidOperationException(
$"Unknown message content match kind '{_kind}'."
$"Unknown message content match kind '{kind}'."
)
};
}
@@ -4,20 +4,14 @@ using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering;
internal class MentionsMessageFilter : MessageFilter
internal class MentionsMessageFilter(string value) : MessageFilter
{
private readonly string _value;
public MentionsMessageFilter(string value) => _value = value;
public override bool IsMatch(Message message) =>
message
.MentionedUsers
.Any(
user =>
string.Equals(_value, user.Name, StringComparison.OrdinalIgnoreCase)
|| string.Equals(_value, user.DisplayName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(_value, user.FullName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(_value, user.Id.ToString(), StringComparison.OrdinalIgnoreCase)
);
message.MentionedUsers.Any(
user =>
string.Equals(value, user.Name, StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, user.DisplayName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, user.FullName, StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, user.Id.ToString(), StringComparison.OrdinalIgnoreCase)
);
}
@@ -8,5 +8,6 @@ internal enum MessageContentMatchKind
Video,
Image,
Sound,
Pin
Pin,
Invite
}
@@ -2,11 +2,7 @@
namespace DiscordChatExporter.Core.Exporting.Filtering;
internal class NegatedMessageFilter : MessageFilter
internal class NegatedMessageFilter(MessageFilter filter) : MessageFilter
{
private readonly MessageFilter _filter;
public NegatedMessageFilter(MessageFilter filter) => _filter = filter;
public override bool IsMatch(Message message) => !_filter.IsMatch(message);
public override bool IsMatch(Message message) => !filter.IsMatch(message);
}
@@ -61,18 +61,28 @@ internal static class FilterGrammar
.IgnoreThen(
Parse.OneOf(
Span.EqualToIgnoreCase("link")
.IgnoreThen(Parse.Return(MessageContentMatchKind.Link)),
.IgnoreThen(Parse.Return(MessageContentMatchKind.Link))
.Try(),
Span.EqualToIgnoreCase("embed")
.IgnoreThen(Parse.Return(MessageContentMatchKind.Embed)),
.IgnoreThen(Parse.Return(MessageContentMatchKind.Embed))
.Try(),
Span.EqualToIgnoreCase("file")
.IgnoreThen(Parse.Return(MessageContentMatchKind.File)),
.IgnoreThen(Parse.Return(MessageContentMatchKind.File))
.Try(),
Span.EqualToIgnoreCase("video")
.IgnoreThen(Parse.Return(MessageContentMatchKind.Video)),
.IgnoreThen(Parse.Return(MessageContentMatchKind.Video))
.Try(),
Span.EqualToIgnoreCase("image")
.IgnoreThen(Parse.Return(MessageContentMatchKind.Image)),
.IgnoreThen(Parse.Return(MessageContentMatchKind.Image))
.Try(),
Span.EqualToIgnoreCase("sound")
.IgnoreThen(Parse.Return(MessageContentMatchKind.Sound)),
Span.EqualToIgnoreCase("pin").IgnoreThen(Parse.Return(MessageContentMatchKind.Pin))
Span.EqualToIgnoreCase("pin")
.IgnoreThen(Parse.Return(MessageContentMatchKind.Pin))
.Try(),
Span.EqualToIgnoreCase("invite")
.IgnoreThen(Parse.Return(MessageContentMatchKind.Invite))
.Try()
)
)
.Select(k => (MessageFilter)new HasMessageFilter(k))
@@ -4,23 +4,13 @@ using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering;
internal class ReactionMessageFilter : MessageFilter
internal class ReactionMessageFilter(string value) : MessageFilter
{
private readonly string _value;
public ReactionMessageFilter(string value) => _value = value;
public override bool IsMatch(Message message) =>
message
.Reactions
.Any(
r =>
string.Equals(
_value,
r.Emoji.Id?.ToString(),
StringComparison.OrdinalIgnoreCase
)
|| string.Equals(_value, r.Emoji.Name, StringComparison.OrdinalIgnoreCase)
|| string.Equals(_value, r.Emoji.Code, StringComparison.OrdinalIgnoreCase)
);
message.Reactions.Any(
r =>
string.Equals(value, r.Emoji.Id?.ToString(), StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, r.Emoji.Name, StringComparison.OrdinalIgnoreCase)
|| string.Equals(value, r.Emoji.Code, StringComparison.OrdinalIgnoreCase)
);
}
@@ -18,16 +18,12 @@ internal partial class HtmlMarkdownVisitor(
bool isJumbo
) : MarkdownVisitor
{
private readonly ExportContext _context = context;
private readonly StringBuilder _buffer = buffer;
private readonly bool _isJumbo = isJumbo;
protected override ValueTask VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default
)
{
_buffer.Append(HtmlEncode(text.Text));
buffer.Append(HtmlEncode(text.Text));
return default;
}
@@ -92,9 +88,9 @@ internal partial class HtmlMarkdownVisitor(
)
};
_buffer.Append(openingTag);
buffer.Append(openingTag);
await VisitAsync(formatting.Children, cancellationToken);
_buffer.Append(closingTag);
buffer.Append(closingTag);
}
protected override async ValueTask VisitHeadingAsync(
@@ -102,14 +98,14 @@ internal partial class HtmlMarkdownVisitor(
CancellationToken cancellationToken = default
)
{
_buffer.Append(
buffer.Append(
// lang=html
$"<h{heading.Level}>"
);
await VisitAsync(heading.Children, cancellationToken);
_buffer.Append(
buffer.Append(
// lang=html
$"</h{heading.Level}>"
);
@@ -120,14 +116,14 @@ internal partial class HtmlMarkdownVisitor(
CancellationToken cancellationToken = default
)
{
_buffer.Append(
buffer.Append(
// lang=html
"<ul>"
);
await VisitAsync(list.Items, cancellationToken);
_buffer.Append(
buffer.Append(
// lang=html
"</ul>"
);
@@ -138,14 +134,14 @@ internal partial class HtmlMarkdownVisitor(
CancellationToken cancellationToken = default
)
{
_buffer.Append(
buffer.Append(
// lang=html
"<li>"
);
await VisitAsync(listItem.Children, cancellationToken);
_buffer.Append(
buffer.Append(
// lang=html
"</li>"
);
@@ -156,7 +152,7 @@ internal partial class HtmlMarkdownVisitor(
CancellationToken cancellationToken = default
)
{
_buffer.Append(
buffer.Append(
// lang=html
$"""
<code class="chatlog__markdown-pre chatlog__markdown-pre--inline">{HtmlEncode(inlineCodeBlock.Code)}</code>
@@ -175,7 +171,7 @@ internal partial class HtmlMarkdownVisitor(
? $"language-{multiLineCodeBlock.Language}"
: "nohighlight";
_buffer.Append(
buffer.Append(
// lang=html
$"""
<code class="chatlog__markdown-pre chatlog__markdown-pre--multiline {highlightClass}">{HtmlEncode(multiLineCodeBlock.Code)}</code>
@@ -196,7 +192,7 @@ internal partial class HtmlMarkdownVisitor(
.Groups[1]
.Value;
_buffer.Append(
buffer.Append(
!string.IsNullOrWhiteSpace(linkedMessageId)
// lang=html
? $"""<a href="{HtmlEncode(link.Url)}" onclick="scrollToMessage(event, '{linkedMessageId}')">"""
@@ -206,7 +202,7 @@ internal partial class HtmlMarkdownVisitor(
await VisitAsync(link.Children, cancellationToken);
_buffer.Append(
buffer.Append(
// lang=html
"</a>"
);
@@ -218,9 +214,9 @@ internal partial class HtmlMarkdownVisitor(
)
{
var emojiImageUrl = Emoji.GetImageUrl(emoji.Id, emoji.Name, emoji.IsAnimated);
var jumboClass = _isJumbo ? "chatlog__emoji--large" : "";
var jumboClass = isJumbo ? "chatlog__emoji--large" : "";
_buffer.Append(
buffer.Append(
// lang=html
$"""
<img
@@ -228,7 +224,7 @@ internal partial class HtmlMarkdownVisitor(
class="chatlog__emoji {jumboClass}"
alt="{emoji.Name}"
title="{emoji.Code}"
src="{await _context.ResolveAssetUrlAsync(emojiImageUrl, cancellationToken)}">
src="{await context.ResolveAssetUrlAsync(emojiImageUrl, cancellationToken)}">
"""
);
}
@@ -240,7 +236,7 @@ internal partial class HtmlMarkdownVisitor(
{
if (mention.Kind == MentionKind.Everyone)
{
_buffer.Append(
buffer.Append(
// lang=html
"""
<span class="chatlog__markdown-mention">@everyone</span>
@@ -249,7 +245,7 @@ internal partial class HtmlMarkdownVisitor(
}
else if (mention.Kind == MentionKind.Here)
{
_buffer.Append(
buffer.Append(
// lang=html
"""
<span class="chatlog__markdown-mention">@here</span>
@@ -262,13 +258,13 @@ internal partial class HtmlMarkdownVisitor(
// which means they need to be populated on demand.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/304
if (mention.TargetId is not null)
await _context.PopulateMemberAsync(mention.TargetId.Value, cancellationToken);
await context.PopulateMemberAsync(mention.TargetId.Value, cancellationToken);
var member = mention.TargetId?.Pipe(_context.TryGetMember);
var member = mention.TargetId?.Pipe(context.TryGetMember);
var fullName = member?.User.FullName ?? "Unknown";
var displayName = member?.DisplayName ?? member?.User.DisplayName ?? "Unknown";
_buffer.Append(
buffer.Append(
// lang=html
$"""
<span class="chatlog__markdown-mention" title="{HtmlEncode(fullName)}">@{HtmlEncode(displayName)}</span>
@@ -277,11 +273,11 @@ internal partial class HtmlMarkdownVisitor(
}
else if (mention.Kind == MentionKind.Channel)
{
var channel = mention.TargetId?.Pipe(_context.TryGetChannel);
var channel = mention.TargetId?.Pipe(context.TryGetChannel);
var symbol = channel?.IsVoice == true ? "🔊" : "#";
var name = channel?.Name ?? "deleted-channel";
_buffer.Append(
buffer.Append(
// lang=html
$"""
<span class="chatlog__markdown-mention">{symbol}{HtmlEncode(name)}</span>
@@ -290,7 +286,7 @@ internal partial class HtmlMarkdownVisitor(
}
else if (mention.Kind == MentionKind.Role)
{
var role = mention.TargetId?.Pipe(_context.TryGetRole);
var role = mention.TargetId?.Pipe(context.TryGetRole);
var name = role?.Name ?? "deleted-role";
var color = role?.Color;
@@ -300,7 +296,7 @@ internal partial class HtmlMarkdownVisitor(
"""
: null;
_buffer.Append(
buffer.Append(
// lang=html
$"""
<span class="chatlog__markdown-mention" style="{style}">@{HtmlEncode(name)}</span>
@@ -315,14 +311,14 @@ internal partial class HtmlMarkdownVisitor(
)
{
var formatted = timestamp.Instant is not null
? _context.FormatDate(timestamp.Instant.Value, timestamp.Format ?? "g")
? context.FormatDate(timestamp.Instant.Value, timestamp.Format ?? "g")
: "Invalid date";
var formattedLong = timestamp.Instant is not null
? _context.FormatDate(timestamp.Instant.Value, "f")
? context.FormatDate(timestamp.Instant.Value, "f")
: "";
_buffer.Append(
buffer.Append(
// lang=html
$"""
<span class="chatlog__markdown-timestamp" title="{HtmlEncode(formattedLong)}">{HtmlEncode(formatted)}</span>
@@ -13,7 +13,6 @@ internal class HtmlMessageWriter(Stream stream, ExportContext context, string th
: MessageWriter(stream, context)
{
private readonly TextWriter _writer = new StreamWriter(stream);
private readonly string _themeName = themeName;
private readonly HtmlMinifier _minifier = new();
private readonly List<Message> _messageGroup = new();
@@ -74,11 +73,9 @@ internal class HtmlMessageWriter(Stream stream, ExportContext context, string th
{
await _writer.WriteLineAsync(
Minify(
await new PreambleTemplate
{
Context = Context,
ThemeName = _themeName
}.RenderAsync(cancellationToken)
await new PreambleTemplate { Context = Context, ThemeName = themeName }.RenderAsync(
cancellationToken
)
)
);
}
@@ -15,18 +15,19 @@ namespace DiscordChatExporter.Core.Exporting;
internal class JsonMessageWriter(Stream stream, ExportContext context)
: MessageWriter(stream, context)
{
private readonly Utf8JsonWriter _writer = new Utf8JsonWriter(
stream,
new JsonWriterOptions
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/450
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
Indented = true,
// Validation errors may mask actual failures
// https://github.com/Tyrrrz/DiscordChatExporter/issues/413
SkipValidation = true
}
);
private readonly Utf8JsonWriter _writer =
new(
stream,
new JsonWriterOptions
{
// https://github.com/Tyrrrz/DiscordChatExporter/issues/450
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
Indented = true,
// Validation errors may mask actual failures
// https://github.com/Tyrrrz/DiscordChatExporter/issues/413
SkipValidation = true
}
);
private async ValueTask<string> FormatMarkdownAsync(
string markdown,
@@ -445,14 +446,12 @@ internal class JsonMessageWriter(Stream stream, ExportContext context)
_writer.WriteStartArray("users");
await foreach (
var user in Context
.Discord
.GetMessageReactionsAsync(
Context.Request.Channel.Id,
message.Id,
reaction.Emoji,
cancellationToken
)
var user in Context.Discord.GetMessageReactionsAsync(
Context.Request.Channel.Id,
message.Id,
reaction.Emoji,
cancellationToken
)
)
{
_writer.WriteStartObject();
@@ -8,8 +8,6 @@ namespace DiscordChatExporter.Core.Exporting;
internal partial class MessageExporter(ExportContext context) : IAsyncDisposable
{
private readonly ExportContext _context = context;
private int _partitionIndex;
private MessageWriter? _writer;
@@ -39,10 +37,10 @@ internal partial class MessageExporter(ExportContext context) : IAsyncDisposable
// Ensure that the partition limit has not been reached
if (
_writer is not null
&& _context
.Request
.PartitionLimit
.IsReached(_writer.MessagesWritten, _writer.BytesWritten)
&& context.Request.PartitionLimit.IsReached(
_writer.MessagesWritten,
_writer.BytesWritten
)
)
{
await ResetWriterAsync(cancellationToken);
@@ -53,10 +51,10 @@ internal partial class MessageExporter(ExportContext context) : IAsyncDisposable
if (_writer is not null)
return _writer;
Directory.CreateDirectory(_context.Request.OutputDirPath);
var filePath = GetPartitionFilePath(_context.Request.OutputFilePath, _partitionIndex);
Directory.CreateDirectory(context.Request.OutputDirPath);
var filePath = GetPartitionFilePath(context.Request.OutputFilePath, _partitionIndex);
var writer = CreateMessageWriter(filePath, _context.Request.Format, _context);
var writer = CreateMessageWriter(filePath, context.Request.Format, context);
await writer.WritePreambleAsync(cancellationToken);
return _writer = writer;
@@ -448,7 +448,10 @@
</div>
}
// Generic video embed
else if (embed.Kind == EmbedKind.Video && !string.IsNullOrWhiteSpace(embed.Url))
else if (embed.Kind == EmbedKind.Video
&& !string.IsNullOrWhiteSpace(embed.Url)
// Twitch clips cannot be embedded in local HTML files
&& embed.TryGetTwitchClip() is null)
{
var embedVideoUrl =
embed.Video?.ProxyUrl ?? embed.Video?.Url ??
@@ -1,11 +1,7 @@
namespace DiscordChatExporter.Core.Exporting.Partitioning;
internal class FileSizePartitionLimit : PartitionLimit
internal class FileSizePartitionLimit(long limit) : PartitionLimit
{
private readonly long _limit;
public FileSizePartitionLimit(long limit) => _limit = limit;
public override bool IsReached(long messagesWritten, long bytesWritten) =>
bytesWritten >= _limit;
bytesWritten >= limit;
}
@@ -1,11 +1,7 @@
namespace DiscordChatExporter.Core.Exporting.Partitioning;
internal class MessageCountPartitionLimit : PartitionLimit
internal class MessageCountPartitionLimit(long limit) : PartitionLimit
{
private readonly long _limit;
public MessageCountPartitionLimit(long limit) => _limit = limit;
public override bool IsReached(long messagesWritten, long bytesWritten) =>
messagesWritten >= _limit;
messagesWritten >= limit;
}
@@ -1,7 +1,6 @@
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Markdown.Parsing;
using DiscordChatExporter.Core.Utils.Extensions;
@@ -11,15 +10,12 @@ namespace DiscordChatExporter.Core.Exporting;
internal partial class PlainTextMarkdownVisitor(ExportContext context, StringBuilder buffer)
: MarkdownVisitor
{
private readonly ExportContext _context = context;
private readonly StringBuilder _buffer = buffer;
protected override ValueTask VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default
)
{
_buffer.Append(text.Text);
buffer.Append(text.Text);
return default;
}
@@ -28,7 +24,7 @@ internal partial class PlainTextMarkdownVisitor(ExportContext context, StringBui
CancellationToken cancellationToken = default
)
{
_buffer.Append(emoji.IsCustomEmoji ? $":{emoji.Name}:" : emoji.Name);
buffer.Append(emoji.IsCustomEmoji ? $":{emoji.Name}:" : emoji.Name);
return default;
}
@@ -40,11 +36,11 @@ internal partial class PlainTextMarkdownVisitor(ExportContext context, StringBui
{
if (mention.Kind == MentionKind.Everyone)
{
_buffer.Append("@everyone");
buffer.Append("@everyone");
}
else if (mention.Kind == MentionKind.Here)
{
_buffer.Append("@here");
buffer.Append("@here");
}
else if (mention.Kind == MentionKind.User)
{
@@ -52,30 +48,30 @@ internal partial class PlainTextMarkdownVisitor(ExportContext context, StringBui
// which means they need to be populated on demand.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/304
if (mention.TargetId is not null)
await _context.PopulateMemberAsync(mention.TargetId.Value, cancellationToken);
await context.PopulateMemberAsync(mention.TargetId.Value, cancellationToken);
var member = mention.TargetId?.Pipe(_context.TryGetMember);
var member = mention.TargetId?.Pipe(context.TryGetMember);
var displayName = member?.DisplayName ?? member?.User.DisplayName ?? "Unknown";
_buffer.Append($"@{displayName}");
buffer.Append($"@{displayName}");
}
else if (mention.Kind == MentionKind.Channel)
{
var channel = mention.TargetId?.Pipe(_context.TryGetChannel);
var channel = mention.TargetId?.Pipe(context.TryGetChannel);
var name = channel?.Name ?? "deleted-channel";
_buffer.Append($"#{name}");
buffer.Append($"#{name}");
// Voice channel marker
if (channel?.IsVoice == true)
_buffer.Append(" [voice]");
buffer.Append(" [voice]");
}
else if (mention.Kind == MentionKind.Role)
{
var role = mention.TargetId?.Pipe(_context.TryGetRole);
var role = mention.TargetId?.Pipe(context.TryGetRole);
var name = role?.Name ?? "deleted-role";
_buffer.Append($"@{name}");
buffer.Append($"@{name}");
}
}
@@ -84,9 +80,9 @@ internal partial class PlainTextMarkdownVisitor(ExportContext context, StringBui
CancellationToken cancellationToken = default
)
{
_buffer.Append(
buffer.Append(
timestamp.Instant is not null
? _context.FormatDate(timestamp.Instant.Value, timestamp.Format ?? "g")
? context.FormatDate(timestamp.Instant.Value, timestamp.Format ?? "g")
: "Invalid date"
);
@@ -702,6 +702,10 @@
.chatlog__embed-spotify {
border: 0;
}
.chatlog__embed-twitch {
border: 0;
}
.chatlog__embed-youtube-container {
margin-top: 0.6rem;
@@ -8,5 +8,5 @@ internal record LinkNode(string Url, IReadOnlyList<MarkdownNode> Children)
IContainerNode
{
public LinkNode(string url)
: this(url, new[] { new TextNode(url) }) { }
: this(url, [new TextNode(url)]) { }
}
@@ -2,15 +2,8 @@
namespace DiscordChatExporter.Core.Markdown.Parsing;
internal class AggregateMatcher<T> : IMatcher<T>
internal class AggregateMatcher<T>(IReadOnlyList<IMatcher<T>> matchers) : IMatcher<T>
{
private readonly IReadOnlyList<IMatcher<T>> _matchers;
public AggregateMatcher(IReadOnlyList<IMatcher<T>> matchers)
{
_matchers = matchers;
}
public AggregateMatcher(params IMatcher<T>[] matchers)
: this((IReadOnlyList<IMatcher<T>>)matchers) { }
@@ -19,7 +12,7 @@ internal class AggregateMatcher<T> : IMatcher<T>
ParsedMatch<T>? earliestMatch = null;
// Try to match the input with each matcher and get the match with the lowest start index
foreach (var matcher in _matchers)
foreach (var matcher in matchers)
{
// Try to match
var match = matcher.TryMatch(segment);
@@ -1,14 +1,8 @@
namespace DiscordChatExporter.Core.Markdown.Parsing;
internal class ParsedMatch<T>
internal class ParsedMatch<T>(StringSegment segment, T value)
{
public StringSegment Segment { get; }
public StringSegment Segment { get; } = segment;
public T Value { get; }
public ParsedMatch(StringSegment segment, T value)
{
Segment = segment;
Value = value;
}
public T Value { get; } = value;
}
@@ -3,20 +3,11 @@ using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Markdown.Parsing;
internal class RegexMatcher<T> : IMatcher<T>
internal class RegexMatcher<T>(Regex regex, Func<StringSegment, Match, T?> transform) : IMatcher<T>
{
private readonly Regex _regex;
private readonly Func<StringSegment, Match, T?> _transform;
public RegexMatcher(Regex regex, Func<StringSegment, Match, T?> transform)
{
_regex = regex;
_transform = transform;
}
public ParsedMatch<T>? TryMatch(StringSegment segment)
{
var match = _regex.Match(segment.Source, segment.StartIndex, segment.Length);
var match = regex.Match(segment.Source, segment.StartIndex, segment.Length);
if (!match.Success)
return null;
@@ -25,11 +16,11 @@ internal class RegexMatcher<T> : IMatcher<T>
// Which is super weird because regex.Match(string, int) takes the whole input in context.
// So in order to properly account for ^/$ regex tokens, we need to make sure that
// the expression also matches on the bigger part of the input.
if (!_regex.IsMatch(segment.Source[..segment.EndIndex], segment.StartIndex))
if (!regex.IsMatch(segment.Source[..segment.EndIndex], segment.StartIndex))
return null;
var segmentMatch = segment.Relocate(match);
var value = _transform(segmentMatch, match);
var value = transform(segmentMatch, match);
return value is not null ? new ParsedMatch<T>(segmentMatch, value) : null;
}
@@ -2,37 +2,24 @@
namespace DiscordChatExporter.Core.Markdown.Parsing;
internal class StringMatcher<T> : IMatcher<T>
internal class StringMatcher<T>(
string needle,
StringComparison comparison,
Func<StringSegment, T?> transform
) : IMatcher<T>
{
private readonly string _needle;
private readonly StringComparison _comparison;
private readonly Func<StringSegment, T?> _transform;
public StringMatcher(
string needle,
StringComparison comparison,
Func<StringSegment, T?> transform
)
{
_needle = needle;
_comparison = comparison;
_transform = transform;
}
public StringMatcher(string needle, Func<StringSegment, T> transform)
: this(needle, StringComparison.Ordinal, transform) { }
public ParsedMatch<T>? TryMatch(StringSegment segment)
{
var index = segment
.Source
.IndexOf(_needle, segment.StartIndex, segment.Length, _comparison);
var index = segment.Source.IndexOf(needle, segment.StartIndex, segment.Length, comparison);
if (index < 0)
return null;
var segmentMatch = segment.Relocate(index, _needle.Length);
var value = _transform(segmentMatch);
var segmentMatch = segment.Relocate(index, needle.Length);
var value = transform(segmentMatch);
return value is not null ? new ParsedMatch<T>(segmentMatch, value) : null;
}
+9
View File
@@ -0,0 +1,9 @@
using System;
namespace DiscordChatExporter.Core.Utils;
public static class Docker
{
public static bool IsRunningInContainer { get; } =
Environment.GetEnvironmentVariable("DOTNET_RUNNING_IN_CONTAINER") == "true";
}
@@ -1,5 +1,4 @@
using System;
using System.Collections.Generic;
using System.Text;
namespace DiscordChatExporter.Core.Utils.Extensions;
@@ -27,16 +26,6 @@ public static class StringExtensions
return builder.ToString();
}
public static IEnumerable<Rune> GetRunes(this string str)
{
var lastIndex = 0;
while (lastIndex < str.Length && Rune.TryGetRuneAt(str, lastIndex, out var rune))
{
yield return rune;
lastIndex += rune.Utf16SequenceLength;
}
}
public static T? ParseEnumOrNull<T>(this string str, bool ignoreCase = true)
where T : struct, Enum => Enum.TryParse<T>(str, ignoreCase, out var result) ? result : null;
@@ -2,4 +2,4 @@
namespace DiscordChatExporter.Gui.Behaviors;
public class ChannelMultiSelectionListBoxBehavior : MultiSelectionListBoxBehavior<Channel> { }
public class ChannelMultiSelectionListBoxBehavior : MultiSelectionListBoxBehavior<Channel>;
@@ -10,7 +10,12 @@ public class ChannelToGroupKeyConverter : IValueConverter
{
public static ChannelToGroupKeyConverter Instance { get; } = new();
public object? Convert(object value, Type targetType, object parameter, CultureInfo culture) =>
public object? Convert(
object? value,
Type targetType,
object? parameter,
CultureInfo culture
) =>
value switch
{
Channel { IsThread: true, Parent: not null } channel
@@ -22,9 +27,9 @@ public class ChannelToGroupKeyConverter : IValueConverter
};
public object ConvertBack(
object value,
object? value,
Type targetType,
object parameter,
object? parameter,
CultureInfo culture
) => throw new NotSupportedException();
}
@@ -9,15 +9,20 @@ public class DateTimeOffsetToDateTimeConverter : IValueConverter
{
public static DateTimeOffsetToDateTimeConverter Instance { get; } = new();
public object? Convert(object value, Type targetType, object parameter, CultureInfo culture) =>
public object? Convert(
object? value,
Type targetType,
object? parameter,
CultureInfo culture
) =>
value is DateTimeOffset dateTimeOffsetValue
? dateTimeOffsetValue.DateTime
: default(DateTime?);
public object? ConvertBack(
object value,
object? value,
Type targetType,
object parameter,
object? parameter,
CultureInfo culture
) =>
value is DateTime dateTimeValue
@@ -10,13 +10,17 @@ public class ExportFormatToStringConverter : IValueConverter
{
public static ExportFormatToStringConverter Instance { get; } = new();
public object? Convert(object value, Type targetType, object parameter, CultureInfo culture) =>
value is ExportFormat exportFormatValue ? exportFormatValue.GetDisplayName() : default;
public object? Convert(
object? value,
Type targetType,
object? parameter,
CultureInfo culture
) => value is ExportFormat exportFormatValue ? exportFormatValue.GetDisplayName() : default;
public object ConvertBack(
object value,
object? value,
Type targetType,
object parameter,
object? parameter,
CultureInfo culture
) => throw new NotSupportedException();
}
@@ -9,13 +9,13 @@ public class InverseBoolConverter : IValueConverter
{
public static InverseBoolConverter Instance { get; } = new();
public object Convert(object value, Type targetType, object parameter, CultureInfo culture) =>
public object Convert(object? value, Type targetType, object? parameter, CultureInfo culture) =>
value is false;
public object ConvertBack(
object value,
object? value,
Type targetType,
object parameter,
object? parameter,
CultureInfo culture
) => value is false;
}
@@ -9,13 +9,15 @@ public class LocaleToDisplayNameConverter : IValueConverter
{
public static LocaleToDisplayNameConverter Instance { get; } = new();
public object? Convert(object value, Type targetType, object parameter, CultureInfo culture) =>
value is string locale ? CultureInfo.GetCultureInfo(locale).DisplayName : null;
public object Convert(object? value, Type targetType, object? parameter, CultureInfo culture) =>
value is string locale && !string.IsNullOrWhiteSpace(locale)
? CultureInfo.GetCultureInfo(locale).DisplayName
: "System default";
public object ConvertBack(
object value,
object? value,
Type targetType,
object parameter,
object? parameter,
CultureInfo culture
) => throw new NotSupportedException();
}
@@ -10,13 +10,17 @@ public class SnowflakeToDateTimeOffsetConverter : IValueConverter
{
public static SnowflakeToDateTimeOffsetConverter Instance { get; } = new();
public object? Convert(object value, Type targetType, object parameter, CultureInfo culture) =>
value is Snowflake snowflake ? snowflake.ToDate() : null;
public object? Convert(
object? value,
Type targetType,
object? parameter,
CultureInfo culture
) => value is Snowflake snowflake ? snowflake.ToDate() : null;
public object ConvertBack(
object value,
object? value,
Type targetType,
object parameter,
object? parameter,
CultureInfo culture
) => throw new NotSupportedException();
}
@@ -9,13 +9,17 @@ public class TimeSpanToDateTimeConverter : IValueConverter
{
public static TimeSpanToDateTimeConverter Instance { get; } = new();
public object? Convert(object value, Type targetType, object parameter, CultureInfo culture) =>
value is TimeSpan timeSpanValue ? DateTime.Today.Add(timeSpanValue) : default(DateTime?);
public object? Convert(
object? value,
Type targetType,
object? parameter,
CultureInfo culture
) => value is TimeSpan timeSpanValue ? DateTime.Today.Add(timeSpanValue) : default(DateTime?);
public object? ConvertBack(
object value,
object? value,
Type targetType,
object parameter,
object? parameter,
CultureInfo culture
) => value is DateTime dateTimeValue ? dateTimeValue.TimeOfDay : default(TimeSpan?);
}
@@ -14,9 +14,9 @@
<ItemGroup>
<PackageReference Include="Cogwheel" Version="2.0.4" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.2" PrivateAssets="all" />
<PackageReference Include="CSharpier.MsBuild" Version="0.26.7" PrivateAssets="all" />
<PackageReference Include="Deorcify" Version="1.0.2" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.5.1" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.5.2" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.1.1" />
<PackageReference Include="MaterialDesignColors" Version="2.1.4" />
<PackageReference Include="MaterialDesignThemes" Version="4.9.0" />
@@ -1,5 +1,4 @@
using System;
using System.Globalization;
using System.IO;
using Cogwheel;
using DiscordChatExporter.Core.Exporting;
@@ -8,7 +7,8 @@ using Microsoft.Win32;
namespace DiscordChatExporter.Gui.Services;
public partial class SettingsService : SettingsBase
public partial class SettingsService()
: SettingsBase(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Settings.dat"))
{
public bool IsUkraineSupportMessageEnabled { get; set; } = true;
@@ -20,7 +20,7 @@ public partial class SettingsService : SettingsBase
public ThreadInclusionMode ThreadInclusionMode { get; set; } = ThreadInclusionMode.None;
public string Locale { get; set; } = CultureInfo.CurrentCulture.Name;
public string? Locale { get; set; }
public bool IsUtcNormalizationEnabled { get; set; }
@@ -44,9 +44,6 @@ public partial class SettingsService : SettingsBase
public string? LastAssetsDirPath { get; set; }
public SettingsService()
: base(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Settings.dat")) { }
public override void Save()
{
// Clear the token if it's not supposed to be persisted
@@ -67,8 +64,7 @@ public partial class SettingsService
try
{
return Registry
.CurrentUser
.OpenSubKey(
.CurrentUser.OpenSubKey(
"SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Themes\\Personalize",
false
)
@@ -6,27 +6,20 @@ using Onova.Services;
namespace DiscordChatExporter.Gui.Services;
public class UpdateService : IDisposable
public class UpdateService(SettingsService settingsService) : IDisposable
{
private readonly IUpdateManager _updateManager = new UpdateManager(
new GithubPackageResolver("Tyrrrz", "DiscordChatExporter", "DiscordChatExporter.zip"),
new ZipPackageExtractor()
);
private readonly SettingsService _settingsService;
private Version? _updateVersion;
private bool _updatePrepared;
private bool _updaterLaunched;
public UpdateService(SettingsService settingsService)
{
_settingsService = settingsService;
}
public async ValueTask<Version?> CheckForUpdatesAsync()
{
if (!_settingsService.IsAutoUpdateEnabled)
if (!settingsService.IsAutoUpdateEnabled)
return null;
var check = await _updateManager.CheckForUpdatesAsync();
@@ -35,7 +28,7 @@ public class UpdateService : IDisposable
public async ValueTask PrepareUpdateAsync(Version version)
{
if (!_settingsService.IsAutoUpdateEnabled)
if (!settingsService.IsAutoUpdateEnabled)
return;
try
@@ -55,7 +48,7 @@ public class UpdateService : IDisposable
public void FinalizeUpdate(bool needRestart)
{
if (!_settingsService.IsAutoUpdateEnabled)
if (!settingsService.IsAutoUpdateEnabled)
return;
if (_updateVersion is null || !_updatePrepared || _updaterLaunched)
@@ -235,8 +235,7 @@ public class DashboardViewModel : PropertyChangedBase
var exporter = new ChannelExporter(_discord);
var channelProgressPairs = dialog
.Channels!
.Select(c => new { Channel = c, Progress = _progressMuxer.CreateInput() })
.Channels!.Select(c => new { Channel = c, Progress = _progressMuxer.CreateInput() })
.ToArray();
var successfulExportCount = 0;
@@ -1,33 +1,31 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using DiscordChatExporter.Core.Utils.Extensions;
using DiscordChatExporter.Gui.Models;
using DiscordChatExporter.Gui.Services;
using DiscordChatExporter.Gui.ViewModels.Framework;
namespace DiscordChatExporter.Gui.ViewModels.Dialogs;
public class SettingsViewModel : DialogScreen
public class SettingsViewModel(SettingsService settingsService) : DialogScreen
{
private readonly SettingsService _settingsService;
public bool IsAutoUpdateEnabled
{
get => _settingsService.IsAutoUpdateEnabled;
set => _settingsService.IsAutoUpdateEnabled = value;
get => settingsService.IsAutoUpdateEnabled;
set => settingsService.IsAutoUpdateEnabled = value;
}
public bool IsDarkModeEnabled
{
get => _settingsService.IsDarkModeEnabled;
set => _settingsService.IsDarkModeEnabled = value;
get => settingsService.IsDarkModeEnabled;
set => settingsService.IsDarkModeEnabled = value;
}
public bool IsTokenPersisted
{
get => _settingsService.IsTokenPersisted;
set => _settingsService.IsTokenPersisted = value;
get => settingsService.IsTokenPersisted;
set => settingsService.IsTokenPersisted = value;
}
public IReadOnlyList<ThreadInclusionMode> AvailableThreadInclusions { get; } =
@@ -35,14 +33,15 @@ public class SettingsViewModel : DialogScreen
public ThreadInclusionMode ThreadInclusionMode
{
get => _settingsService.ThreadInclusionMode;
set => _settingsService.ThreadInclusionMode = value;
get => settingsService.ThreadInclusionMode;
set => settingsService.ThreadInclusionMode = value;
}
// These items have to be non-nullable because WPF ComboBox doesn't allow a null value to be selected
public IReadOnlyList<string> AvailableLocales { get; } = new[]
{
// Current locale
CultureInfo.CurrentCulture.Name,
// Current locale (maps to null downstream)
"",
// Locales supported by the Discord app
"da-DK",
"de-DE",
@@ -75,23 +74,23 @@ public class SettingsViewModel : DialogScreen
"ko-KR"
}.Distinct(StringComparer.OrdinalIgnoreCase).ToArray();
// This has to be non-nullable because WPF ComboBox doesn't allow a null value to be selected
public string Locale
{
get => _settingsService.Locale;
set => _settingsService.Locale = value;
get => settingsService.Locale ?? "";
// Important to reduce empty strings to nulls, because empty strings don't correspond to valid cultures
set => settingsService.Locale = value.NullIfWhiteSpace();
}
public bool IsUtcNormalizationEnabled
{
get => _settingsService.IsUtcNormalizationEnabled;
set => _settingsService.IsUtcNormalizationEnabled = value;
get => settingsService.IsUtcNormalizationEnabled;
set => settingsService.IsUtcNormalizationEnabled = value;
}
public int ParallelLimit
{
get => _settingsService.ParallelLimit;
set => _settingsService.ParallelLimit = Math.Clamp(value, 1, 10);
get => settingsService.ParallelLimit;
set => settingsService.ParallelLimit = Math.Clamp(value, 1, 10);
}
public SettingsViewModel(SettingsService settingsService) => _settingsService = settingsService;
}
@@ -8,19 +8,13 @@ using Stylet;
namespace DiscordChatExporter.Gui.ViewModels.Framework;
public class DialogManager : IDisposable
public class DialogManager(IViewManager viewManager) : IDisposable
{
private readonly IViewManager _viewManager;
private readonly SemaphoreSlim _dialogLock = new(1, 1);
public DialogManager(IViewManager viewManager)
{
_viewManager = viewManager;
}
public async ValueTask<T?> ShowDialogAsync<T>(DialogScreen<T> dialogScreen)
{
var view = _viewManager.CreateAndBindViewForModelIfNecessary(dialogScreen);
var view = viewManager.CreateAndBindViewForModelIfNecessary(dialogScreen);
void OnDialogOpened(object? openSender, DialogOpenedEventArgs openArgs)
{
@@ -16,4 +16,4 @@ public abstract class DialogScreen<T> : PropertyChangedBase
}
}
public abstract class DialogScreen : DialogScreen<bool?> { }
public abstract class DialogScreen : DialogScreen<bool?>;
@@ -166,44 +166,55 @@
<Run
FontSize="16"
FontWeight="SemiBold"
Text="To get token for your personal account:" />
Text="To get the token for your personal account:" />
<LineBreak />
<Run Text="* Automating user accounts is technically against TOS —" />
<Run FontWeight="SemiBold" Text="use at your own risk" /><Run Text="!" />
<LineBreak />
<Run Text="1. Open Discord in your" />
<Hyperlink Command="{s:Action OpenDiscord}">
<Run Text="web browser" />
</Hyperlink>
<Run Text="and login" />
<LineBreak />
<Run Text="2. Open any server or direct message channel" />
<LineBreak />
<Run Text="3. Press" />
<Run FontWeight="SemiBold" Text="Ctrl+Shift+I" />
<Run Text="to show developer tools" />
<LineBreak />
<Run Text="4. Navigate to the" />
<Run FontWeight="SemiBold" Text="Network" />
<Run Text="tab" />
<LineBreak />
<Run Text="5. Press" />
<Run FontWeight="SemiBold" Text="Ctrl+R" />
<Run Text="to reload" />
<LineBreak />
<Run Text="6. Switch between random channels to trigger network requests" />
<LineBreak />
<Run Text="7. Search for a request that starts with" />
<Run FontWeight="SemiBold" Text="messages" />
<LineBreak />
<Run Text="8. Select the" />
<Run FontWeight="SemiBold" Text="Headers" />
<Run Text="tab on the right" />
<LineBreak />
<Run Text="9. Scroll down to the" />
<Run FontWeight="SemiBold" Text="Request Headers" />
<Run Text="section" />
<LineBreak />
<Run Text="10. Copy the value of the" />
<Run FontWeight="SemiBold" Text="authorization" />
<Run Text="header" />
@@ -220,25 +231,30 @@
<Run
FontSize="16"
FontWeight="SemiBold"
Text="To get token for a bot account:" />
Text="To get the token for your bot:" />
<LineBreak />
<Run Text="1. Open Discord" />
<Hyperlink Command="{s:Action OpenDiscordDeveloperPortal}">
<Run Text="developer portal" />
</Hyperlink>
<LineBreak />
<Run Text="2. Open your application's settings" />
<LineBreak />
<Run Text="3. Navigate to the" />
<Run FontWeight="SemiBold" Text="Bot" />
<Run Text="section on the left" />
<LineBreak />
<Run Text="4. Under" />
<Run FontWeight="SemiBold" Text="Token" />
<Run Text="click" />
<Run FontWeight="SemiBold" Text="Copy" />
<LineBreak />
<Run Text="* Your bot needs to have" />
<Run Text="* Your bot needs to have the" />
<Run FontWeight="SemiBold" Text="Message Content Intent" />
<Run Text="enabled to read messages" />
<LineBreak />
@@ -89,17 +89,18 @@
<TextBox.ToolTip>
<TextBlock>
<Run Text="Output file or directory path." />
<Run Text="If a directory is specified, file names will be generated automatically based on the channel names and other parameters." />
<Run Text="Directory path should end with a slash to avoid ambiguity." />
<Run Text="If a directory is specified, file names will be generated automatically based on the channel names and export parameters." />
<Run Text="Directory paths must end with a slash to avoid ambiguity." />
<Run Text="Supports template tokens, see the documentation for more info." />
<LineBreak />
<LineBreak />
<Run Text="Available template tokens:" />
<LineBreak />
<Run FontWeight="SemiBold" Text="%g" />
<Run Text="— guild ID" />
<Run Text="— server ID" />
<LineBreak />
<Run FontWeight="SemiBold" Text="%G" />
<Run Text="— guild name" />
<Run Text="— server name" />
<LineBreak />
<Run FontWeight="SemiBold" Text="%t" />
<Run Text="— category ID" />
@@ -232,7 +233,7 @@
materialDesign:HintAssist.IsFloating="True"
Style="{DynamicResource MaterialDesignOutlinedTextBox}"
Text="{Binding MessageFilterValue}"
ToolTip="Only include messages that satisfy this filter (e.g. 'from:foo#1234' or 'has:image')" />
ToolTip="Only include messages that satisfy this filter (e.g. 'from:foo#1234' or 'has:image'). See the documentation for more info." />
<!-- Markdown formatting -->
<Grid Margin="16,8" ToolTip="Process markdown, mentions, and other special tokens">
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2017-2023 Oleksii Holub
Copyright (c) 2017-2024 Oleksii Holub
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+4 -3
View File
@@ -90,12 +90,13 @@ The following table lists all available download options:
</table>
> **Important**:
> To run **DiscordChatExporter** on macOS and Linux, you need to make sure that **.NET 8.0 Runtime** is installed.
> To run **DiscordChatExporter**, you need to make sure that the **.NET 8.0 Runtime** is installed.
> You can download it here:
>
> - [.NET 8.0 Runtime for **macOS x64**](https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-8.0.0-macos-x64-installer)
> - [.NET 8.0 Runtime for **macOS arm64**](https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-8.0.0-macos-arm64-installer)
> - [.NET 8.0 Runtime for **Linux**](https://learn.microsoft.com/dotnet/core/install/linux) (find the correct download for your distro)
> - On Windows, the runtime should be installed automatically when you run the application for the first time
>
> This should not be necessary if you install **DiscordChatExporter** using a package manager, as it should take care of the dependencies for you.
> This is also not necessary if you are running **DiscordChatExporter** via Docker, because the image already contains the runtime.
@@ -108,11 +109,11 @@ The following table lists all available download options:
- Graphical user interface (Windows)
- Command-line interface (Windows, Linux, macOS)
- Authentication via both user and bot tokens
- Authentication via either a user or a bot token
- Multiple output formats: HTML (dark/light), TXT, CSV, JSON
- Support for markdown, attachments, embeds, emoji, and other rich media features
- File partitioning, date ranges, message filtering, and other export options
- Self-contained exports that don't require internet
- Self-contained exports that can be viewed offline
## Screenshots
+10
View File
@@ -0,0 +1,10 @@
#!/usr/bin/env sh
# If we are root, ensure the files in /out are writable
# by the dce user and restart the process as the dce user
if [ "$(id -u)" = '0' ]; then
chown -R dce:dce /out
exec su-exec dce "$0" "$@"
fi
exec /opt/app/DiscordChatExporter.Cli "$@"