Compare commits

...

40 Commits
2.29 ... 2.30.1

Author SHA1 Message Date
Tyrrrz
e849c9516c Update version 2021-08-28 19:51:11 +03:00
Tyrrrz
524d0a1bb2 Fix breaking changes in tests 2021-08-28 19:47:51 +03:00
Tyrrrz
f0a4410ee0 Update NuGet packages 2021-08-28 19:44:02 +03:00
Tyrrrz
a98772e46e Try using ConcurrentDictionary for caching in tests 2021-08-28 19:41:50 +03:00
Tyrrrz
b6fdea11a2 Refactor webhook trigger in CD 2021-08-28 19:27:34 +03:00
Tyrrrz
fcca052165 Ignore unix timestamp markers with invalid offsets
Fixes #681
2021-08-28 19:17:57 +03:00
Chris Helder
ac7ebd6f91 Use higher resolution for user profile pictures (#685) 2021-08-27 05:03:51 -07:00
Lucas LaBuff
9303383662 Fix parsing of "has:file" filter (#678) 2021-08-08 13:49:50 -07:00
Tyrrrz
7fa908ce34 Fix Docker CD workflow 2021-08-08 19:13:15 +03:00
Tyrrrz
345dc568c0 Update version 2021-08-08 18:57:43 +03:00
Alexey Golub
5c0b657e0e Update CD 2021-07-28 15:07:30 -07:00
Tyrrrz
265cb71dfd Add note explaining why Razor templates are not rendered directly 2021-07-27 13:02:53 +03:00
Alexey Golub
0a6fea1ce9 Use powershell for Discord webhook 2021-07-26 15:27:30 -07:00
Tyrrrz
ebb36de354 Trigger Discord webhook on deploy 2021-07-26 19:16:46 +03:00
Tyrrrz
48539084f7 Add mentions of the Discord server 2021-07-26 18:55:46 +03:00
Tyrrrz
9ec95670c5 Update readme 2021-07-26 18:05:34 +03:00
Tyrrrz
a449c02dd5 Authenticate Docker jobs 2021-07-26 18:02:44 +03:00
Tyrrrz
89b1f9924b Switch to manual Docker builds 2021-07-26 17:53:49 +03:00
Tyrrrz
99f94425c2 Stop pluralizing "emoji" as "emojis" 2021-07-26 17:07:02 +03:00
Tyrrrz
ab71b035fc Update tests readme 2021-07-24 18:17:07 +03:00
Tyrrrz
fdc3c1b012 Add test instructions 2021-07-24 18:15:16 +03:00
Tyrrrz
6fa6df981e [HTML] Fix bot tag positioning 2021-07-24 17:46:27 +03:00
Tyrrrz
eec4123b98 [HTML] Add horizontal gap between embed fields 2021-07-24 17:39:45 +03:00
Tyrrrz
6fa7cbe568 Allow link nodes to have markdown children instead of just text
Closes #640
2021-07-24 17:20:28 +03:00
Tyrrrz
aae3790a5f [HTML] Special case plain image embeds
Closes #537
2021-07-23 02:29:05 +03:00
Tyrrrz
28b2039a33 [HTML] Update quote style
Closes #654
2021-07-23 01:28:49 +03:00
Tyrrrz
f40589dea6 [HTML] Special case Spotify embeds
Closes #657
2021-07-23 00:54:00 +03:00
Tyrrrz
bb4db21b87 Add missing formats to the list of supported image and video attachments 2021-07-23 00:18:38 +03:00
Tyrrrz
38bc259ab6 [HTML] Render full date in tooltips on unix timestamps
Closes #656
2021-07-21 19:14:27 +03:00
Tyrrrz
530e2c50a9 Simplify User.GetAvatarUrl(...) 2021-07-21 17:29:05 +03:00
Tyrrrz
63c92047a1 Test improvements 2021-07-19 23:13:52 +03:00
Tyrrrz
7d9d3ca5fb Exclude EmojiIndex from coverage 2021-07-19 22:37:04 +03:00
Tyrrrz
cad6baba48 Fix coverage collection in tests 2021-07-19 22:28:30 +03:00
Tyrrrz
9c25bceabf Fix isolation in tests 2021-07-19 21:07:36 +03:00
Tyrrrz
28de1053f6 Workaround for Spectre parallelization issue in tests 2021-07-19 20:59:48 +03:00
Tyrrrz
ec0494c038 More tests 2021-07-19 20:35:27 +03:00
Tyrrrz
d735d2acd3 Improve tests 2021-07-19 20:19:32 +03:00
Tyrrrz
85d53d0e94 Basic automated tests through the CLI 2021-07-19 20:09:35 +03:00
Tyrrrz
daa8c0a735 [HTML] Update fallback text when referenced message is deleted or could not be loaded
Closes #645
2021-07-19 15:30:18 +03:00
Tyrrrz
d4eba75036 Normalize namings ("...Type" -> "...Kind") 2021-07-19 02:07:19 +03:00
72 changed files with 1631 additions and 346 deletions

View File

@@ -1,5 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: 💬 Discord server
url: https://discord.gg/2SUWKFnHSm
about: Chat with the project community.
- name: 📖 Wiki
url: https://github.com/Tyrrrz/DiscordChatExporter/wiki
about: Find usage guides and frequently asked questions.

29
.github/workflows/CD-Docker.yml vendored Normal file
View File

@@ -0,0 +1,29 @@
name: Docker CD
on:
push:
tags:
- "*"
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2.3.3
- name: Get release version
id: get-version
uses: dawidd6/action-get-tag@v1
- name: Login
run: docker login -u tyrrrz -p ${{ secrets.DOCKER_TOKEN }}
- name: Build
run: docker build -t tyrrrz/discordchatexporter:${{ steps.get-version.outputs.tag }} -t tyrrrz/discordchatexporter:stable .
- name: Deploy
run: |
docker push tyrrrz/discordchatexporter:${{ steps.get-version.outputs.tag }}
docker push tyrrrz/discordchatexporter:stable

View File

@@ -13,6 +13,10 @@ jobs:
- name: Checkout
uses: actions/checkout@v2.3.3
- name: Get release version
id: get-version
uses: dawidd6/action-get-tag@v1
- name: Install .NET (v3.1)
uses: actions/setup-dotnet@v1.7.2
with:
@@ -69,3 +73,17 @@ jobs:
asset_path: DiscordChatExporter.Gui/bin/Publish/Archive.zip
asset_name: DiscordChatExporter.zip
asset_content_type: application/zip
- name: Notify Discord
uses: satak/webrequest-action@v1.2.4
with:
url: ${{ secrets.DISCORD_WEBHOOK }}
method: POST
headers: |
{
"ContentType": "application/json; charset=UTF-8"
}
payload: |
{
"content": "**DiscordChatExporter** new version released!\nVersion: `${{ steps.get-version.outputs.tag }}`\nChangelog: <https://github.com/Tyrrrz/DiscordChatExporter/blob/${{ steps.get-version.outputs.tag }}/Changelog.md>"
}

23
.github/workflows/CI-Docker.yml vendored Normal file
View File

@@ -0,0 +1,23 @@
name: Docker CI
on:
push:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2.3.3
- name: Login
run: docker login -u tyrrrz -p ${{ secrets.DOCKER_TOKEN }}
- name: Build
run: docker build -t tyrrrz/discordchatexporter:latest .
- name: Deploy
run: docker push tyrrrz/discordchatexporter:latest

View File

@@ -20,6 +20,17 @@ jobs:
with:
dotnet-version: 5.0.x
- name: Build & test
run: dotnet test --configuration Release --logger GitHubActions
env:
DISCORD_TOKEN: ${{ secrets.DISCORD_TOKEN }}
DISCORD_TOKEN_BOT: true
- name: Upload coverage
uses: codecov/codecov-action@v1.0.5
with:
token: ${{ secrets.CODECOV_TOKEN }}
- name: Build & publish (CLI)
run: dotnet publish DiscordChatExporter.Cli/ -o DiscordChatExporter.Cli/bin/Publish/ --configuration Release
@@ -36,4 +47,4 @@ jobs:
uses: actions/upload-artifact@v1
with:
name: DiscordChatExporter
path: DiscordChatExporter.Gui/bin/Publish/
path: DiscordChatExporter.Gui/bin/Publish/

5
.gitignore vendored
View File

@@ -19,4 +19,7 @@ bld/
[Oo]bj/
# Coverage
*.opencover.xml
*.opencover.xml
# Secrets
*.secret

View File

@@ -1,3 +1,21 @@
### v2.30.1 (28-Aug-2021)
- [HTML] Changed user avatars to render using higher resolution images. (Thanks [@Chris Helder](https://github.com/TheDude53))
- Fixed an issue where `has:file` message filter was not recognized. (Thanks [@Lucas LaBuff](https://github.com/96-LB))
- Fixed an issue where a unix timestamp marker with an invalid offset crashed the export process.
### v2.30 (08-Aug-2021)
- [HTML] Added special casing for Spotify track embeds to render them directly using an embedded player.
- [HTML] Added special casing for plain image embeds to render them like image attachments.
- [HTML] Added tooltip text for unix timestamp markers which shows the full date represented by the timestamp, regardless of configured date format.
- [HTML] Updated the fallback text shown when the referenced message could not be loaded to make it more explicit.
- [HTML] Updated styling of quoted text in markdown to more closely resemble how it looks in Discord.
- [HTML] Fixed an issue where some file formats were not correctly recognized as image or video attachments, which affected how they were rendered.
- [HTML] Fixed an issue where formatting inside named links (e.g. `[**Bold link**](https://example.com)`) was not being parsed.
- [HTML] Fixed an issue where inline embed fields were missing horizontal spacing between columns.
- [HTML] Fixed other minor styling issues.
### v2.29 (19-Jul-2021)
- Added support for message filtering, which is accessible as an additional export parameter in the GUI and as the `--filter` option in the CLI. Message filters use a special syntax similar to Discord's search query syntax to allow specifying which messages should be included in the export. For example, you can limit the export to only include messages that contain image attachments (`has:image`) or messages from a certain user (`from:Tyrrrz`) or messages containing certain words (`hello world`), etc. You can also combine filters together (`hi mentions:Tyrrrz`, `from:Tyrrrz | from:96-LB`, `(mentions:Tyrrrz | from:Tyrrrz) has:image`) or negate them (`-has:link`). More thorough documentation coming soon. (Thanks [@Lucas LaBuff](https://github.com/96-LB))

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<Version>2.29</Version>
<Version>2.30.1</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Alexey Golub</Copyright>
<LangVersion>preview</LangVersion>

View File

@@ -0,0 +1,31 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>
<CollectCoverage>true</CollectCoverage>
<CoverletOutputFormat>opencover</CoverletOutputFormat>
</PropertyGroup>
<ItemGroup>
<Content Include="xunit.runner.json" CopyToOutputDirectory="PreserveNewest" />
<None Include="*.secret" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="AngleSharp" Version="0.16.0" />
<PackageReference Include="FluentAssertions" Version="6.1.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="1.2.0" />
<PackageReference Include="JsonExtensions" Version="1.1.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.11.0" />
<PackageReference Include="coverlet.msbuild" Version="3.1.0" PrivateAssets="all" />
<PackageReference Include="System.Reactive" Version="5.0.0" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.3" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Cli\DiscordChatExporter.Cli.csproj" />
</ItemGroup>
</Project>

View File

@@ -0,0 +1,210 @@
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Fixtures;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.TestData;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using Xunit;
using Xunit.Abstractions;
namespace DiscordChatExporter.Cli.Tests
{
public class EmbedSpecs : IClassFixture<TempOutputFixture>
{
private readonly ITestOutputHelper _testOutput;
private readonly TempOutputFixture _tempOutput;
public EmbedSpecs(ITestOutputHelper testOutput, TempOutputFixture tempOutput)
{
_testOutput = testOutput;
_tempOutput = tempOutput;
}
[Fact]
public async Task Message_with_an_embed_is_rendered_correctly_in_JSON()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("json");
// Act
var jsonData = await GlobalCache.WrapAsync("embed-specs-output-json", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.EmbedTestCases)},
ExportFormat = ExportFormat.Json,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(jsonData);
var json = Json.Parse(jsonData);
var messageJson = json
.GetProperty("messages")
.EnumerateArray()
.Single(j => string.Equals(
j.GetProperty("id").GetString(),
"866769910729146400",
StringComparison.OrdinalIgnoreCase
));
var embed = messageJson
.GetProperty("embeds")
.EnumerateArray()
.Single();
var embedAuthor = embed.GetProperty("author");
var embedThumbnail = embed.GetProperty("thumbnail");
var embedFooter = embed.GetProperty("footer");
var embedFields = embed.GetProperty("fields").EnumerateArray().ToArray();
// Assert
embed.GetProperty("title").GetString().Should().Be("Embed title");
embed.GetProperty("url").GetString().Should().Be("https://example.com");
embed.GetProperty("timestamp").GetString().Should().Be("2021-07-14T21:00:00+00:00");
embed.GetProperty("description").GetString().Should().Be("**Embed** _description_");
embed.GetProperty("color").GetString().Should().Be("#58B9FF");
embedAuthor.GetProperty("name").GetString().Should().Be("Embed author");
embedAuthor.GetProperty("url").GetString().Should().Be("https://example.com/author");
embedAuthor.GetProperty("iconUrl").GetString().Should().NotBeNullOrWhiteSpace();
embedThumbnail.GetProperty("url").GetString().Should().NotBeNullOrWhiteSpace();
embedThumbnail.GetProperty("width").GetInt32().Should().Be(120);
embedThumbnail.GetProperty("height").GetInt32().Should().Be(120);
embedFooter.GetProperty("text").GetString().Should().Be("Embed footer");
embedFooter.GetProperty("iconUrl").GetString().Should().NotBeNullOrWhiteSpace();
embedFields.Should().HaveCount(3);
embedFields[0].GetProperty("name").GetString().Should().Be("Field 1");
embedFields[0].GetProperty("value").GetString().Should().Be("Value 1");
embedFields[0].GetProperty("isInline").GetBoolean().Should().BeTrue();
embedFields[1].GetProperty("name").GetString().Should().Be("Field 2");
embedFields[1].GetProperty("value").GetString().Should().Be("Value 2");
embedFields[1].GetProperty("isInline").GetBoolean().Should().BeTrue();
embedFields[2].GetProperty("name").GetString().Should().Be("Field 3");
embedFields[2].GetProperty("value").GetString().Should().Be("Value 3");
embedFields[2].GetProperty("isInline").GetBoolean().Should().BeTrue();
}
[Fact]
public async Task Message_with_an_embed_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("embed-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.EmbedTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866769910729146400");
var messageText = messageHtml?.Text();
// Assert
messageText.Should().ContainAll(
"Embed author",
"Embed title",
"Embed description",
"Field 1", "Value 1",
"Field 2", "Value 2",
"Field 3", "Value 3",
"Embed footer"
);
}
[Fact]
public async Task Message_with_a_Spotify_track_is_rendered_using_an_iframe_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("embed-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.EmbedTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-867886632203976775");
var iframeHtml = messageHtml?.QuerySelector("iframe");
// Assert
iframeHtml.Should().NotBeNull();
iframeHtml?.GetAttribute("src").Should()
.StartWithEquivalentOf("https://open.spotify.com/embed/track/1LHZMWefF9502NPfArRfvP");
}
[Fact]
public async Task Message_with_a_YouTube_video_is_rendered_using_an_iframe_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("embed-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.EmbedTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866472508588294165");
var iframeHtml = messageHtml?.QuerySelector("iframe");
// Assert
iframeHtml.Should().NotBeNull();
iframeHtml?.GetAttribute("src").Should()
.StartWithEquivalentOf("https://www.youtube.com/embed/qOWW4OlgbvE");
}
}
}

View File

@@ -0,0 +1,31 @@
using System;
using System.IO;
namespace DiscordChatExporter.Cli.Tests.Fixtures
{
public class TempOutputFixture : IDisposable
{
public string DirPath => Path.Combine(
Path.GetDirectoryName(typeof(TempOutputFixture).Assembly.Location) ?? Directory.GetCurrentDirectory(),
"Temp",
Guid.NewGuid().ToString()
);
public TempOutputFixture() => Directory.CreateDirectory(DirPath);
public string GetTempFilePath() => Path.Combine(DirPath, Guid.NewGuid().ToString());
public string GetTempFilePath(string extension) => Path.ChangeExtension(GetTempFilePath(), extension);
public void Dispose()
{
try
{
Directory.Delete(DirPath, true);
}
catch (DirectoryNotFoundException)
{
}
}
}
}

View File

@@ -0,0 +1,46 @@
using System;
using System.IO;
namespace DiscordChatExporter.Cli.Tests.Infra
{
internal static class Secrets
{
private static readonly Lazy<string> DiscordTokenLazy = new(() =>
{
var fromEnvironment = Environment.GetEnvironmentVariable("DISCORD_TOKEN");
if (!string.IsNullOrWhiteSpace(fromEnvironment))
return fromEnvironment;
var secretFilePath = Path.Combine(
Path.GetDirectoryName(typeof(Secrets).Assembly.Location) ?? Directory.GetCurrentDirectory(),
"DiscordToken.secret"
);
if (File.Exists(secretFilePath))
return File.ReadAllText(secretFilePath);
throw new InvalidOperationException("Discord token not provided for tests.");
});
private static readonly Lazy<bool> IsDiscordTokenBotLazy = new(() =>
{
var fromEnvironment = Environment.GetEnvironmentVariable("DISCORD_TOKEN_BOT");
if (!string.IsNullOrWhiteSpace(fromEnvironment))
return string.Equals(fromEnvironment, "true", StringComparison.OrdinalIgnoreCase);
var secretFilePath = Path.Combine(
Path.GetDirectoryName(typeof(Secrets).Assembly.Location) ?? Directory.GetCurrentDirectory(),
"DiscordTokenBot.secret"
);
if (File.Exists(secretFilePath))
return true;
return false;
});
public static string DiscordToken => DiscordTokenLazy.Value;
public static bool IsDiscordTokenBot => IsDiscordTokenBotLazy.Value;
}
}

View File

@@ -0,0 +1,319 @@
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using AngleSharp.Dom;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Fixtures;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.TestData;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using JsonExtensions;
using Xunit;
using Xunit.Abstractions;
namespace DiscordChatExporter.Cli.Tests
{
public class MentionSpecs : IClassFixture<TempOutputFixture>
{
private readonly ITestOutputHelper _testOutput;
private readonly TempOutputFixture _tempOutput;
public MentionSpecs(ITestOutputHelper testOutput, TempOutputFixture tempOutput)
{
_testOutput = testOutput;
_tempOutput = tempOutput;
}
[Fact]
public async Task User_mention_is_rendered_correctly_in_JSON()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("json");
// Act
var jsonData = await GlobalCache.WrapAsync("mention-specs-output-json", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.Json,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(jsonData);
var json = Json.Parse(jsonData);
var messageJson = json
.GetProperty("messages")
.EnumerateArray()
.Single(j => string.Equals(
j.GetProperty("id").GetString(),
"866458840245076028",
StringComparison.OrdinalIgnoreCase
));
// Assert
messageJson.GetProperty("content").GetString().Should().Be("User mention: @Tyrrrz");
messageJson
.GetProperty("mentions")
.EnumerateArray()
.Select(j => j.GetProperty("id").GetString())
.Should().Contain("128178626683338752");
}
[Fact]
public async Task User_mention_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("mention-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866458840245076028");
// Assert
messageHtml.Should().NotBeNull();
messageHtml?.Text().Trim().Should().Be("User mention: @Tyrrrz");
messageHtml?.InnerHtml.Should().Contain("Tyrrrz#5447");
}
[Fact]
public async Task Text_channel_mention_is_rendered_correctly_in_JSON()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("json");
// Act
var jsonData = await GlobalCache.WrapAsync("mention-specs-output-json", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.Json,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(jsonData);
var json = Json.Parse(jsonData);
var messageJson = json
.GetProperty("messages")
.EnumerateArray()
.Single(j => string.Equals(
j.GetProperty("id").GetString(),
"866459040480624680",
StringComparison.OrdinalIgnoreCase
));
// Assert
messageJson.GetProperty("content").GetString().Should().Be("Text channel mention: #mention-tests");
}
[Fact]
public async Task Text_channel_mention_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("mention-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866459040480624680");
// Assert
messageHtml.Should().NotBeNull();
messageHtml?.Text().Trim().Should().Be("Text channel mention: #mention-tests");
}
[Fact]
public async Task Voice_channel_mention_is_rendered_correctly_in_JSON()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("json");
// Act
var jsonData = await GlobalCache.WrapAsync("mention-specs-output-json", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.Json,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(jsonData);
var json = Json.Parse(jsonData);
var messageJson = json
.GetProperty("messages")
.EnumerateArray()
.Single(j => string.Equals(
j.GetProperty("id").GetString(),
"866459175462633503",
StringComparison.OrdinalIgnoreCase
));
// Assert
messageJson.GetProperty("content").GetString().Should().Be("Voice channel mention: #chaos-vc [voice]");
}
[Fact]
public async Task Voice_channel_mention_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("mention-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866459175462633503");
// Assert
messageHtml.Should().NotBeNull();
messageHtml?.Text().Trim().Should().Be("Voice channel mention: 🔊chaos-vc");
}
[Fact]
public async Task Role_mention_is_rendered_correctly_in_JSON()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("json");
// Act
var jsonData = await GlobalCache.WrapAsync("mention-specs-output-json", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.Json,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(jsonData);
var json = Json.Parse(jsonData);
var messageJson = json
.GetProperty("messages")
.EnumerateArray()
.Single(j => string.Equals(
j.GetProperty("id").GetString(),
"866459254693429258",
StringComparison.OrdinalIgnoreCase
));
// Assert
messageJson.GetProperty("content").GetString().Should().Be("Role mention: @Role 1");
}
[Fact]
public async Task Role_mention_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("mention-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.MentionTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866459254693429258");
// Assert
messageHtml.Should().NotBeNull();
messageHtml?.Text().Trim().Should().Be("Role mention: @Role 1");
}
}
}

View File

@@ -0,0 +1,17 @@
# DiscordChatExporter Tests
This test suite runs against a real Discord server, specifically created to exercise different behaviors required by the test scenarios.
In order to run these tests locally, you need to join the test server and configure your authentication token.
1. Join the server: https://discord.gg/eRV8Vap5bm
2. Locate your Discord authentication token
3. Specify your token using a file or an environment variable:
- **Using a file**: put your token in a new `DiscordToken.secret` file created in this directory
- **Using an environment variable**: set `DISCORD_TOKEN` variable to your token
4. Run the tests: `dotnet test`
> If you're submitting a pull request, you don't _have_ to run the tests locally -- they are executed automatically by CI.
Running them locally can still sometimes be useful for debugging purposes though.
> If you want to have a new test case or a scenario added, please let me know in your pull request.
Currently, it's not possible to add them by yourself.

View File

@@ -0,0 +1,130 @@
using System.IO;
using System.Threading.Tasks;
using AngleSharp.Dom;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands;
using DiscordChatExporter.Cli.Tests.Fixtures;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Cli.Tests.TestData;
using DiscordChatExporter.Cli.Tests.Utils;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Exporting;
using FluentAssertions;
using Xunit;
using Xunit.Abstractions;
namespace DiscordChatExporter.Cli.Tests
{
public class ReplySpecs : IClassFixture<TempOutputFixture>
{
private readonly ITestOutputHelper _testOutput;
private readonly TempOutputFixture _tempOutput;
public ReplySpecs(ITestOutputHelper testOutput, TempOutputFixture tempOutput)
{
_testOutput = testOutput;
_tempOutput = tempOutput;
}
[Fact]
public async Task Reply_to_a_normal_message_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("reply-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.ReplyTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866460738239725598");
// Assert
messageHtml.Should().NotBeNull();
messageHtml?.Text().Trim().Should().Be("reply to original");
messageHtml?.QuerySelector(".chatlog__reference-link")?.Text().Trim().Should().Be("original");
}
[Fact]
public async Task Reply_to_a_deleted_message_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("reply-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.ReplyTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866460975388819486");
// Assert
messageHtml.Should().NotBeNull();
messageHtml?.Text().Trim().Should().Be("reply to deleted");
messageHtml?.QuerySelector(".chatlog__reference-link")?.Text().Trim().Should()
.Be("Original message was deleted or could not be loaded.");
}
[Fact]
public async Task Reply_to_a_empty_message_with_attachment_is_rendered_correctly_in_HTML()
{
// Arrange
var outputFilePath = _tempOutput.GetTempFilePath("html");
// Act
var htmlData = await GlobalCache.WrapAsync("reply-specs-output-html", async () =>
{
await new ExportChannelsCommand
{
TokenValue = Secrets.DiscordToken,
IsBotToken = Secrets.IsDiscordTokenBot,
ChannelIds = new[] {Snowflake.Parse(ChannelIds.ReplyTestCases)},
ExportFormat = ExportFormat.HtmlDark,
OutputPath = outputFilePath
}.ExecuteAsync(new FakeConsole());
return await File.ReadAllTextAsync(outputFilePath);
});
_testOutput.WriteLine(htmlData);
var html = Html.Parse(htmlData);
var messageHtml = html.QuerySelector("#message-866462470335627294");
// Assert
messageHtml.Should().NotBeNull();
messageHtml?.Text().Trim().Should().Be("reply to attachment");
messageHtml?.QuerySelector(".chatlog__reference-link")?.Text().Trim().Should()
.Be("Click to see attachment 🖼️");
}
}
}

View File

@@ -0,0 +1,11 @@
namespace DiscordChatExporter.Cli.Tests.TestData
{
public static class ChannelIds
{
public static string EmbedTestCases => "866472452459462687";
public static string MentionTestCases => "866458801389174794";
public static string ReplyTestCases => "866459871934677052";
}
}

View File

@@ -0,0 +1,22 @@
using System;
using System.Collections.Concurrent;
using System.Threading.Tasks;
namespace DiscordChatExporter.Cli.Tests.Utils
{
internal static class GlobalCache
{
private static readonly ConcurrentDictionary<string, object?> Dictionary = new();
public static async Task<T> WrapAsync<T>(string key, Func<Task<T>> getAsync)
{
if (Dictionary.TryGetValue(key, out var value) && value is T existing)
return existing;
var result = await getAsync();
Dictionary[key] = result;
return result;
}
}
}

View File

@@ -0,0 +1,12 @@
using AngleSharp.Html.Dom;
using AngleSharp.Html.Parser;
namespace DiscordChatExporter.Cli.Tests.Utils
{
internal static class Html
{
private static readonly IHtmlParser Parser = new HtmlParser();
public static IHtmlDocument Parse(string source) => Parser.ParseDocument(source);
}
}

View File

@@ -0,0 +1,5 @@
{
"$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
"methodDisplayOptions": "all",
"methodDisplay": "method"
}

View File

@@ -16,8 +16,8 @@ namespace DiscordChatExporter.Cli.Commands.Base
private AuthToken GetAuthToken() => new(
IsBotToken
? AuthTokenType.Bot
: AuthTokenType.User,
? AuthTokenKind.Bot
: AuthTokenKind.User,
TokenValue
);

View File

@@ -7,14 +7,31 @@ namespace DiscordChatExporter.Cli.Utils.Extensions
{
internal static class ConsoleExtensions
{
public static IAnsiConsole CreateAnsiConsole(this IConsole console) => AnsiConsole.Create(
new AnsiConsoleSettings
{
Ansi = AnsiSupport.Detect,
ColorSystem = ColorSystemSupport.Detect,
Out = new AnsiConsoleOutput(console.Output)
}
);
private class NoopExclusivityMode : IExclusivityMode
{
public T Run<T>(Func<T> func) => func();
public Task<T> Run<T>(Func<Task<T>> func) => func();
}
public static IAnsiConsole CreateAnsiConsole(this IConsole console)
{
// Don't require exclusivity in tests.
// Workaround for https://github.com/spectreconsole/spectre.console/issues/494
var exclusivityMode = console is FakeConsole
? new NoopExclusivityMode()
: null;
return AnsiConsole.Create(
new AnsiConsoleSettings
{
Ansi = AnsiSupport.Detect,
ColorSystem = ColorSystemSupport.Detect,
Out = new AnsiConsoleOutput(console.Output),
ExclusivityMode = exclusivityMode
}
);
}
public static Progress CreateProgressTicker(this IConsole console) => console
.CreateAnsiConsole()

View File

@@ -1,25 +1,27 @@
using System.Net.Http.Headers;
using System.Diagnostics.CodeAnalysis;
using System.Net.Http.Headers;
namespace DiscordChatExporter.Core.Discord
{
public class AuthToken
{
public AuthTokenType Type { get; }
public AuthTokenKind Kind { get; }
public string Value { get; }
public AuthToken(AuthTokenType type, string value)
public AuthToken(AuthTokenKind kind, string value)
{
Type = type;
Kind = kind;
Value = value;
}
public AuthenticationHeaderValue GetAuthenticationHeader() => Type switch
public AuthenticationHeaderValue GetAuthenticationHeader() => Kind switch
{
AuthTokenType.Bot => new AuthenticationHeaderValue("Bot", Value),
AuthTokenKind.Bot => new AuthenticationHeaderValue("Bot", Value),
_ => new AuthenticationHeaderValue(Value)
};
[ExcludeFromCodeCoverage]
public override string ToString() => Value;
}
}

View File

@@ -1,6 +1,6 @@
namespace DiscordChatExporter.Core.Discord
{
public enum AuthTokenType
public enum AuthTokenKind
{
User,
Bot

View File

@@ -1,8 +1,9 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -23,11 +24,11 @@ namespace DiscordChatExporter.Core.Discord.Data
public int? Height { get; }
public bool IsImage => ImageFileExtensions.Contains(FileExtension);
public bool IsImage => FileFormat.IsImage(FileExtension);
public bool IsVideo => VideoFileExtensions.Contains(FileExtension);
public bool IsVideo => FileFormat.IsVideo(FileExtension);
public bool IsAudio => AudioFileExtensions.Contains(FileExtension);
public bool IsAudio => FileFormat.IsAudio(FileExtension);
public bool IsSpoiler => FileName.StartsWith("SPOILER_", StringComparison.Ordinal);
@@ -49,20 +50,12 @@ namespace DiscordChatExporter.Core.Discord.Data
FileSize = fileSize;
}
[ExcludeFromCodeCoverage]
public override string ToString() => FileName;
}
public partial class Attachment
{
private static readonly HashSet<string> ImageFileExtensions = new(StringComparer.OrdinalIgnoreCase)
{".jpg", ".jpeg", ".png", ".gif", ".bmp", ".webp"};
private static readonly HashSet<string> VideoFileExtensions = new(StringComparer.OrdinalIgnoreCase)
{".mp4", ".webm"};
private static readonly HashSet<string> AudioFileExtensions = new(StringComparer.OrdinalIgnoreCase)
{".mp3", ".wav", ".ogg", ".flac", ".m4a"};
public static Attachment Parse(JsonElement json)
{
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);

View File

@@ -1,4 +1,5 @@
using System.Linq;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
@@ -12,14 +13,14 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public Snowflake Id { get; }
public ChannelType Type { get; }
public ChannelKind Kind { get; }
public bool IsTextChannel => Type is
ChannelType.GuildTextChat or
ChannelType.DirectTextChat or
ChannelType.DirectGroupTextChat or
ChannelType.GuildNews or
ChannelType.GuildStore;
public bool IsTextChannel => Kind is
ChannelKind.GuildTextChat or
ChannelKind.DirectTextChat or
ChannelKind.DirectGroupTextChat or
ChannelKind.GuildNews or
ChannelKind.GuildStore;
public bool IsVoiceChannel => !IsTextChannel;
@@ -35,7 +36,7 @@ namespace DiscordChatExporter.Core.Discord.Data
public Channel(
Snowflake id,
ChannelType type,
ChannelKind kind,
Snowflake guildId,
ChannelCategory category,
string name,
@@ -43,7 +44,7 @@ namespace DiscordChatExporter.Core.Discord.Data
string? topic)
{
Id = id;
Type = type;
Kind = kind;
GuildId = guildId;
Category = category;
Name = name;
@@ -51,20 +52,21 @@ namespace DiscordChatExporter.Core.Discord.Data
Topic = topic;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Name;
}
public partial class Channel
{
private static ChannelCategory GetFallbackCategory(ChannelType channelType) => new(
private static ChannelCategory GetFallbackCategory(ChannelKind channelKind) => new(
Snowflake.Zero,
channelType switch
channelKind switch
{
ChannelType.GuildTextChat => "Text",
ChannelType.DirectTextChat => "Private",
ChannelType.DirectGroupTextChat => "Group",
ChannelType.GuildNews => "News",
ChannelType.GuildStore => "Store",
ChannelKind.GuildTextChat => "Text",
ChannelKind.DirectTextChat => "Private",
ChannelKind.DirectGroupTextChat => "Group",
ChannelKind.GuildNews => "News",
ChannelKind.GuildStore => "Store",
_ => "Default"
},
null
@@ -75,7 +77,7 @@ namespace DiscordChatExporter.Core.Discord.Data
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);
var guildId = json.GetPropertyOrNull("guild_id")?.GetString().Pipe(Snowflake.Parse);
var topic = json.GetPropertyOrNull("topic")?.GetString();
var type = (ChannelType) json.GetProperty("type").GetInt32();
var kind = (ChannelKind) json.GetProperty("type").GetInt32();
var name =
// Guild channel
@@ -87,9 +89,9 @@ namespace DiscordChatExporter.Core.Discord.Data
return new Channel(
id,
type,
kind,
guildId ?? Guild.DirectMessages.Id,
category ?? GetFallbackCategory(type),
category ?? GetFallbackCategory(kind),
name,
position ?? json.GetPropertyOrNull("position")?.GetInt32(),
topic

View File

@@ -1,4 +1,5 @@
using System.Text.Json;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -20,6 +21,7 @@ namespace DiscordChatExporter.Core.Discord.Data
Position = position;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Name;
}

View File

@@ -2,7 +2,7 @@
{
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
// Order of enum fields needs to match the order in the docs.
public enum ChannelType
public enum ChannelKind
{
GuildTextChat = 0,
DirectTextChat,

View File

@@ -1,4 +1,5 @@
using System;
using System.Diagnostics.CodeAnalysis;
namespace DiscordChatExporter.Core.Discord.Data.Common
{
@@ -41,6 +42,7 @@ namespace DiscordChatExporter.Core.Discord.Data.Common
return "bytes";
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"{GetLargestWholeNumberValue():0.##} {GetLargestWholeNumberSymbol()}";
}

View File

@@ -1,12 +1,13 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Drawing;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
// https://discord.com/developers/docs/resources/channel#embed-object
public partial class Embed
@@ -55,8 +56,13 @@ namespace DiscordChatExporter.Core.Discord.Data
Footer = footer;
}
public PlainImageEmbedProjection? TryGetPlainImage() => PlainImageEmbedProjection.TryResolve(this);
public SpotifyTrackEmbedProjection? TryGetSpotifyTrack() => SpotifyTrackEmbedProjection.TryResolve(this);
public YouTubeVideoEmbedProjection? TryGetYouTubeVideo() => YouTubeVideoEmbedProjection.TryResolve(this);
[ExcludeFromCodeCoverage]
public override string ToString() => Title ?? "<untitled embed>";
}

View File

@@ -1,7 +1,8 @@
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
// https://discord.com/developers/docs/resources/channel#embed-object-embed-author-structure
public partial class EmbedAuthor
@@ -22,6 +23,7 @@ namespace DiscordChatExporter.Core.Discord.Data
IconProxyUrl = iconProxyUrl;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Name ?? "<unnamed author>";
}

View File

@@ -1,7 +1,8 @@
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
// https://discord.com/developers/docs/resources/channel#embed-object-embed-field-structure
public partial class EmbedField
@@ -19,6 +20,7 @@ namespace DiscordChatExporter.Core.Discord.Data
IsInline = isInline;
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"{Name} | {Value}";
}

View File

@@ -1,7 +1,8 @@
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
// https://discord.com/developers/docs/resources/channel#embed-object-embed-footer-structure
public partial class EmbedFooter
@@ -19,6 +20,7 @@ namespace DiscordChatExporter.Core.Discord.Data
IconProxyUrl = iconProxyUrl;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Text;
}

View File

@@ -1,7 +1,7 @@
using System.Text.Json;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
// https://discord.com/developers/docs/resources/channel#embed-object-embed-image-structure
public partial class EmbedImage

View File

@@ -0,0 +1,45 @@
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
public partial class PlainImageEmbedProjection
{
public string Url { get; }
public PlainImageEmbedProjection(string url) => Url = url;
[ExcludeFromCodeCoverage]
public override string ToString() => Url;
}
public partial class PlainImageEmbedProjection
{
public static PlainImageEmbedProjection? TryResolve(Embed embed)
{
if (string.IsNullOrWhiteSpace(embed.Url))
return null;
// Has to be an embed without any data (except URL and image)
if (!string.IsNullOrWhiteSpace(embed.Title) ||
embed.Timestamp is not null ||
embed.Author is not null ||
!string.IsNullOrWhiteSpace(embed.Description) ||
embed.Fields.Any() ||
embed.Footer is not null)
{
return null;
}
// Has to be an image file
var fileName = Regex.Match(embed.Url, @".+/([^?]*)").Groups[1].Value;
if (string.IsNullOrWhiteSpace(fileName) || !FileFormat.IsImage(Path.GetExtension(fileName)))
return null;
return new PlainImageEmbedProjection(embed.Url);
}
}
}

View File

@@ -0,0 +1,42 @@
using System.Diagnostics.CodeAnalysis;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
public partial class SpotifyTrackEmbedProjection
{
public string TrackId { get; }
public string Url => $"https://open.spotify.com/embed/track/{TrackId}";
public SpotifyTrackEmbedProjection(string trackId) => TrackId = trackId;
[ExcludeFromCodeCoverage]
public override string ToString() => Url;
}
public partial class SpotifyTrackEmbedProjection
{
private static string? TryParseTrackId(string embedUrl)
{
// https://open.spotify.com/track/1LHZMWefF9502NPfArRfvP?si=3efac6ce9be04f0a
var trackId = Regex.Match(embedUrl, @"spotify\.com/track/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(trackId))
return trackId;
return null;
}
public static SpotifyTrackEmbedProjection? TryResolve(Embed embed)
{
if (string.IsNullOrWhiteSpace(embed.Url))
return null;
var trackId = TryParseTrackId(embed.Url);
if (string.IsNullOrWhiteSpace(trackId))
return null;
return new SpotifyTrackEmbedProjection(trackId);
}
}
}

View File

@@ -1,19 +1,17 @@
using System.Text.RegularExpressions;
using System.Diagnostics.CodeAnalysis;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data
namespace DiscordChatExporter.Core.Discord.Data.Embeds
{
public partial class YouTubeVideoEmbedProjection
{
public string VideoId { get; }
public string Url { get; }
public string Url => $"https://www.youtube.com/embed/{VideoId}";
public YouTubeVideoEmbedProjection(string videoId, string url)
{
VideoId = videoId;
Url = url;
}
public YouTubeVideoEmbedProjection(string videoId) => VideoId = videoId;
[ExcludeFromCodeCoverage]
public override string ToString() => Url;
}
@@ -53,9 +51,7 @@ namespace DiscordChatExporter.Core.Discord.Data
if (string.IsNullOrWhiteSpace(videoId))
return null;
var url = $"https://www.youtube.com/embed/{videoId}";
return new YouTubeVideoEmbedProjection(videoId, url);
return new YouTubeVideoEmbedProjection(videoId);
}
}
}

View File

@@ -1,4 +1,5 @@
using System.Linq;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
@@ -33,6 +34,7 @@ namespace DiscordChatExporter.Core.Discord.Data
ImageUrl = imageUrl;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Name;
}

View File

@@ -1,4 +1,5 @@
using System.Text.Json;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
@@ -20,6 +21,7 @@ namespace DiscordChatExporter.Core.Discord.Data
IconUrl = iconUrl;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Name;
}

View File

@@ -1,5 +1,6 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
@@ -26,6 +27,7 @@ namespace DiscordChatExporter.Core.Discord.Data
RoleIds = roleIds;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Nick;
}

View File

@@ -1,15 +1,17 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Core.Discord.Data
{
// https://discord.com/developers/docs/resources/channel#message-object-message-types
public enum MessageType
public enum MessageKind
{
Default = 0,
RecipientAdd = 1,
@@ -27,7 +29,7 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public Snowflake Id { get; }
public MessageType Type { get; }
public MessageKind Kind { get; }
public User Author { get; }
@@ -55,7 +57,7 @@ namespace DiscordChatExporter.Core.Discord.Data
public Message(
Snowflake id,
MessageType type,
MessageKind kind,
User author,
DateTimeOffset timestamp,
DateTimeOffset? editedTimestamp,
@@ -70,7 +72,7 @@ namespace DiscordChatExporter.Core.Discord.Data
Message? referencedMessage)
{
Id = id;
Type = type;
Kind = kind;
Author = author;
Timestamp = timestamp;
EditedTimestamp = editedTimestamp;
@@ -85,6 +87,7 @@ namespace DiscordChatExporter.Core.Discord.Data
ReferencedMessage = referencedMessage;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Content;
}
@@ -97,21 +100,21 @@ namespace DiscordChatExporter.Core.Discord.Data
var timestamp = json.GetProperty("timestamp").GetDateTimeOffset();
var editedTimestamp = json.GetPropertyOrNull("edited_timestamp")?.GetDateTimeOffset();
var callEndedTimestamp = json.GetPropertyOrNull("call")?.GetPropertyOrNull("ended_timestamp")?.GetDateTimeOffset();
var type = (MessageType) json.GetProperty("type").GetInt32();
var kind = (MessageKind) json.GetProperty("type").GetInt32();
var isPinned = json.GetPropertyOrNull("pinned")?.GetBoolean() ?? false;
var messageReference = json.GetPropertyOrNull("message_reference")?.Pipe(MessageReference.Parse);
var referencedMessage = json.GetPropertyOrNull("referenced_message")?.Pipe(Parse);
var content = type switch
var content = kind switch
{
MessageType.RecipientAdd => "Added a recipient.",
MessageType.RecipientRemove => "Removed a recipient.",
MessageType.Call =>
MessageKind.RecipientAdd => "Added a recipient.",
MessageKind.RecipientRemove => "Removed a recipient.",
MessageKind.Call =>
$"Started a call that lasted {callEndedTimestamp?.Pipe(t => t - timestamp).Pipe(t => (int) t.TotalMinutes) ?? 0} minutes.",
MessageType.ChannelNameChange => "Changed the channel name.",
MessageType.ChannelIconChange => "Changed the channel icon.",
MessageType.ChannelPinnedMessage => "Pinned a message.",
MessageType.GuildMemberJoin => "Joined the server.",
MessageKind.ChannelNameChange => "Changed the channel name.",
MessageKind.ChannelIconChange => "Changed the channel icon.",
MessageKind.ChannelPinnedMessage => "Pinned a message.",
MessageKind.GuildMemberJoin => "Joined the server.",
_ => json.GetPropertyOrNull("content")?.GetString() ?? ""
};
@@ -133,7 +136,7 @@ namespace DiscordChatExporter.Core.Discord.Data
return new Message(
id,
type,
kind,
author,
timestamp,
editedTimestamp,

View File

@@ -1,3 +1,4 @@
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -20,6 +21,7 @@ namespace DiscordChatExporter.Core.Discord.Data
GuildId = guildId;
}
[ExcludeFromCodeCoverage]
public override string ToString() => MessageId?.ToString() ?? "<unknown reference>";
}

View File

@@ -1,4 +1,5 @@
using System.Text.Json;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Core.Discord.Data
@@ -16,6 +17,7 @@ namespace DiscordChatExporter.Core.Discord.Data
Count = count;
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"{Emoji} ({Count})";
}

View File

@@ -1,4 +1,5 @@
using System.Drawing;
using System.Diagnostics.CodeAnalysis;
using System.Drawing;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
@@ -29,6 +30,7 @@ namespace DiscordChatExporter.Core.Discord.Data
Color = color;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Name;
}

View File

@@ -1,4 +1,5 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
@@ -37,6 +38,7 @@ namespace DiscordChatExporter.Core.Discord.Data
AvatarUrl = avatarUrl;
}
[ExcludeFromCodeCoverage]
public override string ToString() => FullName;
}
@@ -47,12 +49,11 @@ namespace DiscordChatExporter.Core.Discord.Data
private static string GetAvatarUrl(Snowflake id, string avatarHash)
{
// Animated
if (avatarHash.StartsWith("a_", StringComparison.Ordinal))
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.gif?size=40";
var extension = avatarHash.StartsWith("a_", StringComparison.Ordinal)
? "gif"
: "png";
// Non-animated
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.png?size=40";
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.{extension}?size=128";
}
public static User Parse(JsonElement json)

View File

@@ -126,7 +126,7 @@ namespace DiscordChatExporter.Core.Discord
.ToArray();
var categories = responseOrdered
.Where(j => j.GetProperty("type").GetInt32() == (int) ChannelType.GuildCategory)
.Where(j => j.GetProperty("type").GetInt32() == (int) ChannelKind.GuildCategory)
.Select((j, index) => ChannelCategory.Parse(j, index + 1))
.ToDictionary(j => j.Id.ToString(), StringComparer.Ordinal);

View File

@@ -1,4 +1,5 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
using System.Text.RegularExpressions;
@@ -14,6 +15,7 @@ namespace DiscordChatExporter.Core.Discord
(long) ((Value >> 22) + 1420070400000UL)
).ToLocalTime();
[ExcludeFromCodeCoverage]
public override string ToString() => Value.ToString(CultureInfo.InvariantCulture);
}

View File

@@ -6,7 +6,7 @@
<ItemGroup>
<PackageReference Include="JsonExtensions" Version="1.1.0" />
<PackageReference Include="MiniRazor.CodeGen" Version="2.1.2" />
<PackageReference Include="MiniRazor.CodeGen" Version="2.1.4" />
<PackageReference Include="Polly" Version="7.2.2" />
<PackageReference Include="Superpower" Version="2.3.0" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.5" />

View File

@@ -54,6 +54,7 @@ namespace DiscordChatExporter.Core.Exporting.Filtering.Parsing
.IgnoreThen(Choice(
Span.EqualToIgnoreCase("link").IgnoreThen(Parse.Return(MessageContentMatchKind.Link)),
Span.EqualToIgnoreCase("embed").IgnoreThen(Parse.Return(MessageContentMatchKind.Embed)),
Span.EqualToIgnoreCase("file").IgnoreThen(Parse.Return(MessageContentMatchKind.File)),
Span.EqualToIgnoreCase("video").IgnoreThen(Parse.Return(MessageContentMatchKind.Video)),
Span.EqualToIgnoreCase("image").IgnoreThen(Parse.Return(MessageContentMatchKind.Image)),
Span.EqualToIgnoreCase("sound").IgnoreThen(Parse.Return(MessageContentMatchKind.Sound))

View File

@@ -81,7 +81,7 @@
else
{
<span class="chatlog__reference-unknown">
Original message was deleted.
Original message was deleted or could not be loaded.
</span>
}
</div>
@@ -176,157 +176,239 @@
@{/* Embeds */}
@foreach (var embed in message.Embeds)
{
var youTubeVideo = embed.TryGetYouTubeVideo();
// Plain image embed
if (embed.TryGetPlainImage() is { } plainImageEmbed)
{
<div class="chatlog__embed">
<a href="@await ResolveUrlAsync(plainImageEmbed.Url)">
<img class="chatlog__embed-plainimage" src="@await ResolveUrlAsync(plainImageEmbed.Url)" alt="Embedded image" loading="lazy">
</a>
</div>
}
// Spotify embed
else if (embed.TryGetSpotifyTrack() is { } spotifyTrackEmbed)
{
<div class="chatlog__embed">
<div class="chatlog__embed-spotify-container">
<iframe class="chatlog__embed-spotify" src="@spotifyTrackEmbed.Url" width="400" height="80" allowtransparency="true" allow="encrypted-media"></iframe>
</div>
</div>
}
// YouTube embed
else if (embed.TryGetYouTubeVideo() is { } youTubeVideoEmbed)
{
<div class="chatlog__embed">
@{/* Color pill */}
@if (embed.Color is not null)
{
<div class="chatlog__embed-color-pill" style="background-color: rgba(@embed.Color?.R,@embed.Color?.G,@embed.Color?.B,@embed.Color?.A)"></div>
}
else
{
<div class="chatlog__embed-color-pill chatlog__embed-color-pill--default"></div>
}
<div class="chatlog__embed">
@{/* Color pill */}
@if (embed.Color is not null)
{
<div class="chatlog__embed-color-pill" style="background-color: rgba(@embed.Color?.R,@embed.Color?.G,@embed.Color?.B,@embed.Color?.A)"></div>
}
else
{
<div class="chatlog__embed-color-pill chatlog__embed-color-pill--default"></div>
}
<div class="chatlog__embed-content-container">
<div class="chatlog__embed-content">
<div class="chatlog__embed-text">
@{/* Embed author */}
@if (embed.Author is not null)
{
<div class="chatlog__embed-author">
@if (!string.IsNullOrWhiteSpace(embed.Author.IconUrl))
{
<img class="chatlog__embed-author-icon" src="@await ResolveUrlAsync(embed.Author.IconProxyUrl ?? embed.Author.IconUrl)" alt="Author icon" loading="lazy" onerror="this.style.visibility='hidden'">
}
<div class="chatlog__embed-content-container">
<div class="chatlog__embed-content">
<div class="chatlog__embed-text">
@{/* Embed author */}
@if (embed.Author is not null)
{
<div class="chatlog__embed-author">
@if (!string.IsNullOrWhiteSpace(embed.Author.IconUrl))
{
<img class="chatlog__embed-author-icon" src="@await ResolveUrlAsync(embed.Author.IconProxyUrl ?? embed.Author.IconUrl)" alt="Author icon" loading="lazy" onerror="this.style.visibility='hidden'">
}
@if (!string.IsNullOrWhiteSpace(embed.Author.Name))
{
<span class="chatlog__embed-author-name">
@if (!string.IsNullOrWhiteSpace(embed.Author.Url))
{
<a class="chatlog__embed-author-name-link" href="@embed.Author.Url">@embed.Author.Name</a>
}
else
{
@embed.Author.Name
}
</span>
}
</div>
}
@if (!string.IsNullOrWhiteSpace(embed.Author.Name))
{
<span class="chatlog__embed-author-name">
@if (!string.IsNullOrWhiteSpace(embed.Author.Url))
{
<a class="chatlog__embed-author-name-link" href="@embed.Author.Url">@embed.Author.Name</a>
}
else
{
@embed.Author.Name
}
</span>
}
</div>
}
@{/* Embed title */}
@if (!string.IsNullOrWhiteSpace(embed.Title))
{
<div class="chatlog__embed-title">
@if (!string.IsNullOrWhiteSpace(embed.Url))
{
<a class="chatlog__embed-title-link" href="@embed.Url">
@{/* Embed title */}
@if (!string.IsNullOrWhiteSpace(embed.Title))
{
<div class="chatlog__embed-title">
@if (!string.IsNullOrWhiteSpace(embed.Url))
{
<a class="chatlog__embed-title-link" href="@embed.Url">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Title))</div>
</a>
}
else
{
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Title))</div>
</a>
}
else
{
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Title))</div>
}
</div>
}
}
</div>
}
@{/* Embed description (with special casing for YouTube videos) */}
@if (youTubeVideo is not null)
{
@{/* Video player */}
<div class="chatlog__embed-youtube-container">
<iframe class="chatlog__embed-youtube" src="@youTubeVideo.Url" width="400" height="225"></iframe>
<iframe class="chatlog__embed-youtube" src="@youTubeVideoEmbed.Url" width="400" height="225"></iframe>
</div>
}
else if (!string.IsNullOrWhiteSpace(embed.Description))
{
<div class="chatlog__embed-description">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Description))</div>
</div>
}
</div>
</div>
</div>
</div>
}
// Generic embed
else
{
<div class="chatlog__embed">
@{/* Color pill */}
@if (embed.Color is not null)
{
<div class="chatlog__embed-color-pill" style="background-color: rgba(@embed.Color?.R,@embed.Color?.G,@embed.Color?.B,@embed.Color?.A)"></div>
}
else
{
<div class="chatlog__embed-color-pill chatlog__embed-color-pill--default"></div>
}
@{/* Embed fields */}
@if (embed.Fields.Any())
{
<div class="chatlog__embed-fields">
@foreach (var field in embed.Fields)
{
<div class="chatlog__embed-field @(field.IsInline ? "chatlog__embed-field--inline" : null)">
@if (!string.IsNullOrWhiteSpace(field.Name))
{
<div class="chatlog__embed-field-name">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(field.Name))</div>
</div>
}
<div class="chatlog__embed-content-container">
<div class="chatlog__embed-content">
<div class="chatlog__embed-text">
@{/* Embed author */}
@if (embed.Author is not null)
{
<div class="chatlog__embed-author">
@if (!string.IsNullOrWhiteSpace(embed.Author.IconUrl))
{
<img class="chatlog__embed-author-icon" src="@await ResolveUrlAsync(embed.Author.IconProxyUrl ?? embed.Author.IconUrl)" alt="Author icon" loading="lazy" onerror="this.style.visibility='hidden'">
}
@if (!string.IsNullOrWhiteSpace(field.Value))
{
<div class="chatlog__embed-field-value">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(field.Value))</div>
</div>
}
</div>
}
@if (!string.IsNullOrWhiteSpace(embed.Author.Name))
{
<span class="chatlog__embed-author-name">
@if (!string.IsNullOrWhiteSpace(embed.Author.Url))
{
<a class="chatlog__embed-author-name-link" href="@embed.Author.Url">@embed.Author.Name</a>
}
else
{
@embed.Author.Name
}
</span>
}
</div>
}
@{/* Embed title */}
@if (!string.IsNullOrWhiteSpace(embed.Title))
{
<div class="chatlog__embed-title">
@if (!string.IsNullOrWhiteSpace(embed.Url))
{
<a class="chatlog__embed-title-link" href="@embed.Url">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Title))</div>
</a>
}
else
{
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Title))</div>
}
</div>
}
@{/* Embed description */}
@if (!string.IsNullOrWhiteSpace(embed.Description))
{
<div class="chatlog__embed-description">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Description))</div>
</div>
}
@{/* Embed fields */}
@if (embed.Fields.Any())
{
<div class="chatlog__embed-fields">
@foreach (var field in embed.Fields)
{
<div class="chatlog__embed-field @(field.IsInline ? "chatlog__embed-field--inline" : null)">
@if (!string.IsNullOrWhiteSpace(field.Name))
{
<div class="chatlog__embed-field-name">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(field.Name))</div>
</div>
}
@if (!string.IsNullOrWhiteSpace(field.Value))
{
<div class="chatlog__embed-field-value">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(field.Value))</div>
</div>
}
</div>
}
</div>
}
</div>
@{/* Embed content */}
@if (embed.Thumbnail is not null && !string.IsNullOrWhiteSpace(embed.Thumbnail.Url))
{
<div class="chatlog__embed-thumbnail-container">
<a class="chatlog__embed-thumbnail-link" href="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)">
<img class="chatlog__embed-thumbnail" src="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)" alt="Thumbnail" loading="lazy">
</a>
</div>
}
</div>
@{/* Embed content (not shown for YouTube videos) */}
@if (embed.Thumbnail is not null && !string.IsNullOrWhiteSpace(embed.Thumbnail.Url) && youTubeVideo is null)
@{/* Embed image */}
@if (embed.Image is not null && !string.IsNullOrWhiteSpace(embed.Image.Url))
{
<div class="chatlog__embed-thumbnail-container">
<a class="chatlog__embed-thumbnail-link" href="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)">
<img class="chatlog__embed-thumbnail" src="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)" alt="Thumbnail" loading="lazy">
<div class="chatlog__embed-image-container">
<a class="chatlog__embed-image-link" href="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)">
<img class="chatlog__embed-image" src="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)" alt="Image" loading="lazy">
</a>
</div>
}
@{/* Embed footer & icon */}
@if (embed.Footer is not null || embed.Timestamp is not null)
{
<div class="chatlog__embed-footer">
@{/* Footer icon */}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.IconUrl))
{
<img class="chatlog__embed-footer-icon" src="@await ResolveUrlAsync(embed.Footer.IconProxyUrl ?? embed.Footer.IconUrl)" alt="Footer icon" loading="lazy">
}
<span class="chatlog__embed-footer-text">
@{/* Footer text */}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.Text))
{
@embed.Footer.Text
}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.Text) && embed.Timestamp is not null)
{
@(" • ")
}
@{/* Embed timestamp */}
@if (embed.Timestamp is not null)
{
@FormatDate(embed.Timestamp.Value)
}
</span>
</div>
}
</div>
@{/* Embed image */}
@if (embed.Image is not null && !string.IsNullOrWhiteSpace(embed.Image.Url))
{
<div class="chatlog__embed-image-container">
<a class="chatlog__embed-image-link" href="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)">
<img class="chatlog__embed-image" src="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)" alt="Image" loading="lazy">
</a>
</div>
}
@{/* Embed footer & icon */}
@if (embed.Footer is not null || embed.Timestamp is not null)
{
<div class="chatlog__embed-footer">
@{/* Footer icon */}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.IconUrl))
{
<img class="chatlog__embed-footer-icon" src="@await ResolveUrlAsync(embed.Footer.IconProxyUrl ?? embed.Footer.IconUrl)" alt="Footer icon" loading="lazy">
}
<span class="chatlog__embed-footer-text">
@{/* Footer text */}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.Text))
{
@embed.Footer.Text
}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.Text) && embed.Timestamp is not null)
{
@(" • ")
}
@{/* Embed timestamp */}
@if (embed.Timestamp is not null)
{
@FormatDate(embed.Timestamp.Value)
}
</span>
</div>
}
</div>
</div>
}
}
@{/* Message reactions */}

View File

@@ -107,9 +107,14 @@
}
.quote {
margin: 0.1em 0;
padding-left: 0.6em;
border-left: 4px solid @Themed("#4f545c", "#c7ccd1");
display: flex;
margin: 0.05em 0;
}
.quote::before {
content: "";
margin-right: 0.5em;
border: 2px solid @Themed("#4f545c", "#c7ccd1");
border-radius: 3px;
}
@@ -239,7 +244,7 @@
}
.chatlog__reference-name {
margin-right: 0.25em;
margin-right: 0.3em;
font-weight: 600;
}
@@ -494,6 +499,7 @@
.chatlog__embed-fields {
display: flex;
flex-wrap: wrap;
gap: 0 0.5em;
}
.chatlog__embed-field {
@@ -558,6 +564,17 @@
font-weight: 500;
}
.chatlog__embed-plainimage {
vertical-align: top;
max-width: 45vw;
max-height: 500px;
border-radius: 3px;
}
.chatlog__embed-spotify {
border: 0;
}
.chatlog__embed-youtube-container {
margin-top: 0.6em;
}
@@ -589,11 +606,10 @@
.chatlog__bot-tag {
position: relative;
top: -.2em;
top: -.1em;
margin-left: 0.3em;
padding: 0.05em 0.3em;
border-radius: 3px;
vertical-align: middle;
line-height: 1.3;
background-color: #5865F2;
color: #ffffff;

View File

@@ -25,6 +25,8 @@ namespace DiscordChatExporter.Core.Exporting.Writers
{
var templateContext = new PreambleTemplateContext(Context, _themeName);
// We are not writing directly to output because Razor
// does not actually do asynchronous writes to stream.
await _writer.WriteLineAsync(
await PreambleTemplate.RenderAsync(templateContext)
);
@@ -34,6 +36,8 @@ namespace DiscordChatExporter.Core.Exporting.Writers
{
var templateContext = new MessageGroupTemplateContext(Context, messageGroup);
// We are not writing directly to output because Razor
// does not actually do asynchronous writes to stream.
await _writer.WriteLineAsync(
await MessageGroupTemplate.RenderAsync(templateContext)
);
@@ -66,6 +70,8 @@ namespace DiscordChatExporter.Core.Exporting.Writers
var templateContext = new PostambleTemplateContext(Context, MessagesWritten);
// We are not writing directly to output because Razor
// does not actually do asynchronous writes to stream.
await _writer.WriteLineAsync(
await PostambleTemplate.RenderAsync(templateContext)
);

View File

@@ -3,6 +3,7 @@ using System.Text.Encodings.Web;
using System.Text.Json;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Writing;
@@ -192,7 +193,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
// Channel
_writer.WriteStartObject("channel");
_writer.WriteString("id", Context.Request.Channel.Id.ToString());
_writer.WriteString("type", Context.Request.Channel.Type.ToString());
_writer.WriteString("type", Context.Request.Channel.Kind.ToString());
_writer.WriteString("categoryId", Context.Request.Channel.Category.Id.ToString());
_writer.WriteString("category", Context.Request.Channel.Category.Name);
_writer.WriteString("name", Context.Request.Channel.Name);
@@ -218,7 +219,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
// Metadata
_writer.WriteString("id", message.Id.ToString());
_writer.WriteString("type", message.Type.ToString());
_writer.WriteString("type", message.Kind.ToString());
_writer.WriteString("timestamp", message.Timestamp);
_writer.WriteString("timestampEdited", message.EditedTimestamp);
_writer.WriteString("callEndedTimestamp", message.CallEndedTimestamp);

View File

@@ -30,22 +30,22 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
return base.VisitText(text);
}
protected override MarkdownNode VisitFormatted(FormattedNode formatted)
protected override MarkdownNode VisitFormatting(FormattingNode formatting)
{
var (tagOpen, tagClose) = formatted.Formatting switch
var (tagOpen, tagClose) = formatting.Kind switch
{
TextFormatting.Bold => ("<strong>", "</strong>"),
TextFormatting.Italic => ("<em>", "</em>"),
TextFormatting.Underline => ("<u>", "</u>"),
TextFormatting.Strikethrough => ("<s>", "</s>"),
TextFormatting.Spoiler => (
FormattingKind.Bold => ("<strong>", "</strong>"),
FormattingKind.Italic => ("<em>", "</em>"),
FormattingKind.Underline => ("<u>", "</u>"),
FormattingKind.Strikethrough => ("<s>", "</s>"),
FormattingKind.Spoiler => (
"<span class=\"spoiler-text spoiler-text--hidden\" onclick=\"showSpoiler(event, this)\">", "</span>"),
TextFormatting.Quote => ("<div class=\"quote\">", "</div>"),
_ => throw new ArgumentOutOfRangeException(nameof(formatted.Formatting))
FormattingKind.Quote => ("<div class=\"quote\">", "</div>"),
_ => throw new ArgumentOutOfRangeException(nameof(formatting.Kind))
};
_buffer.Append(tagOpen);
var result = base.VisitFormatted(formatted);
var result = base.VisitFormatting(formatting);
_buffer.Append(tagClose);
return result;
@@ -77,25 +77,22 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
protected override MarkdownNode VisitLink(LinkNode link)
{
// Extract message ID if the link points to a Discord message
var linkedMessageId = Regex.Match(link.Url, "^https?://(?:discord|discordapp).com/channels/.*?/(\\d+)/?$").Groups[1].Value;
// Try to extract message ID if the link refers to a Discord message
var linkedMessageId = Regex.Match(
link.Url,
"^https?://(?:discord|discordapp).com/channels/.*?/(\\d+)/?$"
).Groups[1].Value;
if (!string.IsNullOrWhiteSpace(linkedMessageId))
{
_buffer
.Append($"<a href=\"{Uri.EscapeUriString(link.Url)}\" onclick=\"scrollToMessage(event, '{linkedMessageId}')\">")
.Append(HtmlEncode(link.Title))
.Append("</a>");
}
else
{
_buffer
.Append($"<a href=\"{Uri.EscapeUriString(link.Url)}\">")
.Append(HtmlEncode(link.Title))
.Append("</a>");
}
_buffer.Append(
!string.IsNullOrWhiteSpace(linkedMessageId)
? $"<a href=\"{Uri.EscapeUriString(link.Url)}\" onclick=\"scrollToMessage(event, '{linkedMessageId}')\">"
: $"<a href=\"{Uri.EscapeUriString(link.Url)}\">"
);
return base.VisitLink(link);
var result = base.VisitLink(link);
_buffer.Append("</a>");
return result;
}
protected override MarkdownNode VisitEmoji(EmojiNode emoji)
@@ -162,8 +159,11 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
protected override MarkdownNode VisitUnixTimestamp(UnixTimestampNode timestamp)
{
// Timestamp tooltips always use full date regardless of the configured format
var longDateString = timestamp.Value.ToLocalString("dddd, MMMM d, yyyy h:mm tt");
_buffer
.Append("<span class=\"timestamp\">")
.Append($"<span class=\"timestamp\" title=\"{HtmlEncode(longDateString)}\">")
.Append(HtmlEncode(_context.FormatDate(timestamp.Value)))
.Append("</span>");

View File

@@ -3,6 +3,7 @@ using System.IO;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Discord.Data.Embeds;
using DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors;
using Tyrrrz.Extensions;

View File

@@ -1,4 +1,5 @@
using DiscordChatExporter.Core.Utils;
using System.Diagnostics.CodeAnalysis;
using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Markdown
{
@@ -31,6 +32,7 @@ namespace DiscordChatExporter.Core.Markdown
{
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"<Emoji> {Name}";
}
}

View File

@@ -1,19 +0,0 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown
{
internal class FormattedNode : MarkdownNode
{
public TextFormatting Formatting { get; }
public IReadOnlyList<MarkdownNode> Children { get; }
public FormattedNode(TextFormatting formatting, IReadOnlyList<MarkdownNode> children)
{
Formatting = formatting;
Children = children;
}
public override string ToString() => $"<{Formatting}> (+{Children.Count})";
}
}

View File

@@ -1,6 +1,6 @@
namespace DiscordChatExporter.Core.Markdown
{
internal enum TextFormatting
internal enum FormattingKind
{
Bold,
Italic,

View File

@@ -0,0 +1,29 @@
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
namespace DiscordChatExporter.Core.Markdown
{
internal class FormattingNode : MarkdownNode
{
public FormattingKind Kind { get; }
public IReadOnlyList<MarkdownNode> Children { get; }
public FormattingNode(FormattingKind kind, IReadOnlyList<MarkdownNode> children)
{
Kind = kind;
Children = children;
}
[ExcludeFromCodeCoverage]
public override string ToString()
{
var childrenFormatted = Children.Count == 1
? Children.Single().ToString()
: "+" + Children.Count;
return $"<{Kind}> ({childrenFormatted})";
}
}
}

View File

@@ -1,4 +1,6 @@
namespace DiscordChatExporter.Core.Markdown
using System.Diagnostics.CodeAnalysis;
namespace DiscordChatExporter.Core.Markdown
{
internal class InlineCodeBlockNode : MarkdownNode
{
@@ -9,6 +11,7 @@
Code = code;
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"<Code> {Code}";
}
}

View File

@@ -1,22 +1,34 @@
namespace DiscordChatExporter.Core.Markdown
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
namespace DiscordChatExporter.Core.Markdown
{
internal class LinkNode : MarkdownNode
{
public string Url { get; }
public string Title { get; }
public IReadOnlyList<MarkdownNode> Children { get; }
public LinkNode(string url, string title)
public LinkNode(string url, IReadOnlyList<MarkdownNode> children)
{
Url = url;
Title = title;
Children = children;
}
public LinkNode(string url)
: this(url, url)
: this(url, new[] {new TextNode(url)})
{
}
public override string ToString() => $"<Link> {Title}";
[ExcludeFromCodeCoverage]
public override string ToString()
{
var childrenFormatted = Children.Count == 1
? Children.Single().ToString()
: "+" + Children.Count;
return $"<Link> ({childrenFormatted})";
}
}
}

View File

@@ -1,4 +1,6 @@
namespace DiscordChatExporter.Core.Markdown
using System.Diagnostics.CodeAnalysis;
namespace DiscordChatExporter.Core.Markdown
{
internal class MentionNode : MarkdownNode
{
@@ -12,6 +14,7 @@
Kind = kind;
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"<{Kind} mention> {Id}";
}
}

View File

@@ -1,4 +1,6 @@
namespace DiscordChatExporter.Core.Markdown
using System.Diagnostics.CodeAnalysis;
namespace DiscordChatExporter.Core.Markdown
{
internal class MultiLineCodeBlockNode : MarkdownNode
{
@@ -12,6 +14,7 @@
Code = code;
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"<{Language}> {Code}";
}
}

View File

@@ -7,7 +7,10 @@ using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Markdown.Parsing
{
// The following parsing logic is meant to replicate Discord's markdown grammar as close as possible
// Discord does NOT use a recursive-descent parser for markdown which becomes evident in some
// scenarios, like when multiple formatting nodes are nested together.
// To replicate Discord's behavior, we're employing a special parser that uses a set of regular
// expressions that are executed sequentially in a first-match-first-serve manner.
internal static partial class MarkdownParser
{
private const RegexOptions DefaultRegexOptions =
@@ -18,64 +21,64 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
/* Formatting */
// Capture any character until the earliest double asterisk not followed by an asterisk
private static readonly IMatcher<MarkdownNode> BoldFormattedNodeMatcher = new RegexMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> BoldFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\*\\*(.+?)\\*\\*(?!\\*)", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Bold, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Bold, Parse(p.Slice(m.Groups[1])))
);
// Capture any character until the earliest single asterisk not preceded or followed by an asterisk
// Opening asterisk must not be followed by whitespace
// Closing asterisk must not be preceded by whitespace
private static readonly IMatcher<MarkdownNode> ItalicFormattedNodeMatcher = new RegexMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> ItalicFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\*(?!\\s)(.+?)(?<!\\s|\\*)\\*(?!\\*)", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Italic, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Italic, Parse(p.Slice(m.Groups[1])))
);
// Capture any character until the earliest triple asterisk not followed by an asterisk
private static readonly IMatcher<MarkdownNode> ItalicBoldFormattedNodeMatcher = new RegexMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> ItalicBoldFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\*(\\*\\*.+?\\*\\*)\\*(?!\\*)", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Italic, Parse(p.Slice(m.Groups[1]), BoldFormattedNodeMatcher))
(p, m) => new FormattingNode(FormattingKind.Italic, Parse(p.Slice(m.Groups[1]), BoldFormattingNodeMatcher))
);
// Capture any character except underscore until an underscore
// Closing underscore must not be followed by a word character
private static readonly IMatcher<MarkdownNode> ItalicAltFormattedNodeMatcher = new RegexMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> ItalicAltFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("_([^_]+)_(?!\\w)", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Italic, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Italic, Parse(p.Slice(m.Groups[1])))
);
// Capture any character until the earliest double underscore not followed by an underscore
private static readonly IMatcher<MarkdownNode> UnderlineFormattedNodeMatcher = new RegexMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> UnderlineFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("__(.+?)__(?!_)", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Underline, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Underline, Parse(p.Slice(m.Groups[1])))
);
// Capture any character until the earliest triple underscore not followed by an underscore
private static readonly IMatcher<MarkdownNode> ItalicUnderlineFormattedNodeMatcher =
private static readonly IMatcher<MarkdownNode> ItalicUnderlineFormattingNodeMatcher =
new RegexMatcher<MarkdownNode>(
new Regex("_(__.+?__)_(?!_)", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Italic,
Parse(p.Slice(m.Groups[1]), UnderlineFormattedNodeMatcher))
(p, m) => new FormattingNode(FormattingKind.Italic,
Parse(p.Slice(m.Groups[1]), UnderlineFormattingNodeMatcher))
);
// Capture any character until the earliest double tilde
private static readonly IMatcher<MarkdownNode> StrikethroughFormattedNodeMatcher =
private static readonly IMatcher<MarkdownNode> StrikethroughFormattingNodeMatcher =
new RegexMatcher<MarkdownNode>(
new Regex("~~(.+?)~~", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Strikethrough, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Strikethrough, Parse(p.Slice(m.Groups[1])))
);
// Capture any character until the earliest double pipe
private static readonly IMatcher<MarkdownNode> SpoilerFormattedNodeMatcher = new RegexMatcher<MarkdownNode>(
private static readonly IMatcher<MarkdownNode> SpoilerFormattingNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\|\\|(.+?)\\|\\|", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Spoiler, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Spoiler, Parse(p.Slice(m.Groups[1])))
);
// Capture any character until the end of the line
// Opening 'greater than' character must be followed by whitespace
private static readonly IMatcher<MarkdownNode> SingleLineQuoteNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("^>\\s(.+\n?)", DefaultRegexOptions),
(p, m) => new FormattedNode(TextFormatting.Quote, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Quote, Parse(p.Slice(m.Groups[1])))
);
// Repeatedly capture any character until the end of the line
@@ -86,7 +89,7 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
(_, m) =>
{
var content = string.Concat(m.Groups[1].Captures.Select(c => c.Value));
return new FormattedNode(TextFormatting.Quote, Parse(content));
return new FormattingNode(FormattingKind.Quote, Parse(content));
}
);
@@ -94,7 +97,7 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
// Opening 'greater than' characters must be followed by whitespace
private static readonly IMatcher<MarkdownNode> MultiLineQuoteNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("^>>>\\s(.+)", DefaultRegexOptions | RegexOptions.Singleline),
(p, m) => new FormattedNode(TextFormatting.Quote, Parse(p.Slice(m.Groups[1])))
(p, m) => new FormattingNode(FormattingKind.Quote, Parse(p.Slice(m.Groups[1])))
);
/* Code blocks */
@@ -147,13 +150,13 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
(_, m) => new MentionNode(m.Groups[1].Value, MentionKind.Role)
);
/* Emojis */
/* Emoji */
// Capture any country flag emoji (two regional indicator surrogate pairs)
// ... or "miscellaneous symbol" character
// ... or surrogate pair
// ... or digit followed by enclosing mark
// (this does not match all emojis in Discord but it's reasonably accurate enough)
// (this does not match all emoji in Discord but it's reasonably accurate enough)
private static readonly IMatcher<MarkdownNode> StandardEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("((?:[\\uD83C][\\uDDE6-\\uDDFF]){2}|[\\u2600-\\u26FF]|\\p{Cs}{2}|\\d\\p{Me})", DefaultRegexOptions),
(_, m) => new EmojiNode(m.Groups[1].Value)
@@ -165,7 +168,7 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
(_, m) =>
{
var name = EmojiIndex.TryGetName(m.Groups[1].Value);
return name is not null
return !string.IsNullOrWhiteSpace(name)
? new EmojiNode(name)
: null;
}
@@ -182,10 +185,11 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
// Capture [title](link)
private static readonly IMatcher<MarkdownNode> TitledLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\[(.+?)\\]\\((.+?)\\)", DefaultRegexOptions),
(_, m) => new LinkNode(m.Groups[2].Value, m.Groups[1].Value)
(p, m) => new LinkNode(m.Groups[2].Value, Parse(p.Slice(m.Groups[1])))
);
// Capture any non-whitespace character after http:// or https:// until the last punctuation character or whitespace
// Capture any non-whitespace character after http:// or https://
// until the last punctuation character or whitespace
private static readonly IMatcher<MarkdownNode> AutoLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(https?://\\S*[^\\.,:;\"\'\\s])", DefaultRegexOptions),
(_, m) => new LinkNode(m.Groups[1].Value)
@@ -199,14 +203,14 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
/* Text */
// Capture the shrug emoticon
// Capture the shrug kaomoji
// This escapes it from matching for formatting
private static readonly IMatcher<MarkdownNode> ShrugTextNodeMatcher = new StringMatcher<MarkdownNode>(
@"¯\_(ツ)_/¯",
p => new TextNode(p.ToString())
);
// Capture some specific emojis that don't get rendered
// Capture some specific emoji that don't get rendered
// This escapes it from matching for emoji
private static readonly IMatcher<MarkdownNode> IgnoredEmojiTextNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(\\u26A7|\\u2640|\\u2642|\\u2695|\\u267E|\\u00A9|\\u00AE|\\u2122)", DefaultRegexOptions),
@@ -234,8 +238,8 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
new Regex("<t:(\\d+)(?::\\w)?>", DefaultRegexOptions),
(_, m) =>
{
// We don't care about the 'R' parameter because we're not going to
// show relative timestamps in an export anyway.
// TODO: support formatting parameters
// See: https://github.com/Tyrrrz/DiscordChatExporter/issues/662
if (!long.TryParse(m.Groups[1].Value, NumberStyles.Integer, CultureInfo.InvariantCulture,
out var offset))
@@ -243,6 +247,13 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
return null;
}
// Bound check
// https://github.com/Tyrrrz/DiscordChatExporter/issues/681
if (offset < TimeSpan.MinValue.TotalSeconds || offset > TimeSpan.MaxValue.TotalSeconds)
{
return null;
}
return new UnixTimestampNode(DateTimeOffset.UnixEpoch + TimeSpan.FromSeconds(offset));
}
);
@@ -257,14 +268,14 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
EscapedCharacterTextNodeMatcher,
// Formatting
ItalicBoldFormattedNodeMatcher,
ItalicUnderlineFormattedNodeMatcher,
BoldFormattedNodeMatcher,
ItalicFormattedNodeMatcher,
UnderlineFormattedNodeMatcher,
ItalicAltFormattedNodeMatcher,
StrikethroughFormattedNodeMatcher,
SpoilerFormattedNodeMatcher,
ItalicBoldFormattingNodeMatcher,
ItalicUnderlineFormattingNodeMatcher,
BoldFormattingNodeMatcher,
ItalicFormattingNodeMatcher,
UnderlineFormattingNodeMatcher,
ItalicAltFormattingNodeMatcher,
StrikethroughFormattingNodeMatcher,
SpoilerFormattingNodeMatcher,
MultiLineQuoteNodeMatcher,
RepeatedSingleLineQuoteNodeMatcher,
SingleLineQuoteNodeMatcher,

View File

@@ -8,10 +8,10 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
protected virtual MarkdownNode VisitText(TextNode text) =>
text;
protected virtual MarkdownNode VisitFormatted(FormattedNode formatted)
protected virtual MarkdownNode VisitFormatting(FormattingNode formatting)
{
Visit(formatted.Children);
return formatted;
Visit(formatting.Children);
return formatting;
}
protected virtual MarkdownNode VisitInlineCodeBlock(InlineCodeBlockNode inlineCodeBlock) =>
@@ -20,8 +20,11 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
protected virtual MarkdownNode VisitMultiLineCodeBlock(MultiLineCodeBlockNode multiLineCodeBlock) =>
multiLineCodeBlock;
protected virtual MarkdownNode VisitLink(LinkNode link) =>
link;
protected virtual MarkdownNode VisitLink(LinkNode link)
{
Visit(link.Children);
return link;
}
protected virtual MarkdownNode VisitEmoji(EmojiNode emoji) =>
emoji;
@@ -35,7 +38,7 @@ namespace DiscordChatExporter.Core.Markdown.Parsing
public MarkdownNode Visit(MarkdownNode node) => node switch
{
TextNode text => VisitText(text),
FormattedNode formatted => VisitFormatted(formatted),
FormattingNode formatting => VisitFormatting(formatting),
InlineCodeBlockNode inlineCodeBlock => VisitInlineCodeBlock(inlineCodeBlock),
MultiLineCodeBlockNode multiLineCodeBlock => VisitMultiLineCodeBlock(multiLineCodeBlock),
LinkNode link => VisitLink(link),

View File

@@ -1,4 +1,6 @@
namespace DiscordChatExporter.Core.Markdown
using System.Diagnostics.CodeAnalysis;
namespace DiscordChatExporter.Core.Markdown
{
internal class TextNode : MarkdownNode
{
@@ -9,6 +11,7 @@
Text = text;
}
[ExcludeFromCodeCoverage]
public override string ToString() => Text;
}
}

View File

@@ -1,4 +1,5 @@
using System;
using System.Diagnostics.CodeAnalysis;
namespace DiscordChatExporter.Core.Markdown
{
@@ -8,6 +9,7 @@ namespace DiscordChatExporter.Core.Markdown
public UnixTimestampNode(DateTimeOffset value) => Value = value;
[ExcludeFromCodeCoverage]
public override string ToString() => Value.ToString();
}
}

View File

@@ -1,9 +1,11 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
namespace DiscordChatExporter.Core.Utils
{
// Data sourced from: https://github.com/Tyrrrz/DiscordChatExporter/issues/599#issuecomment-863431045
[ExcludeFromCodeCoverage]
internal static class EmojiIndex
{
private static Dictionary<string, string> _toCodes = new(5000, StringComparer.Ordinal)

View File

@@ -1,4 +1,5 @@
using System;
using System.Diagnostics.CodeAnalysis;
using Superpower;
using Superpower.Parsers;
@@ -12,7 +13,9 @@ namespace DiscordChatExporter.Core.Utils.Extensions
public static TextParser<T> Token<T>(this TextParser<T> parser) =>
parser.Between(Character.WhiteSpace.IgnoreMany(), Character.WhiteSpace.IgnoreMany());
// From: https://twitter.com/nblumhardt/status/1389349059786264578
// Only used for debugging while writing Superpower parsers.
// From https://twitter.com/nblumhardt/status/1389349059786264578
[ExcludeFromCodeCoverage]
public static TextParser<T> Log<T>(this TextParser<T> parser, string description) => i =>
{
Console.WriteLine($"Trying {description} ->");

View File

@@ -0,0 +1,41 @@
using System;
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Utils
{
public static class FileFormat
{
private static readonly HashSet<string> ImageFormats = new(StringComparer.OrdinalIgnoreCase)
{
".jpg",
".jpeg",
".png",
".gif",
".gifv",
".bmp",
".webp"
};
public static bool IsImage(string format) => ImageFormats.Contains(format);
private static readonly HashSet<string> VideoFormats = new(StringComparer.OrdinalIgnoreCase)
{
".mp4",
".webm",
".mov"
};
public static bool IsVideo(string format) => VideoFormats.Contains(format);
private static readonly HashSet<string> AudioFormats = new(StringComparer.OrdinalIgnoreCase)
{
".mp3",
".wav",
".ogg",
".flac",
".m4a"
};
public static bool IsAudio(string format) => AudioFormats.Contains(format);
}
}

View File

@@ -111,7 +111,7 @@ namespace DiscordChatExporter.Gui.ViewModels
if (_settingsService.LastToken is not null)
{
IsBotToken = _settingsService.LastToken.Type == AuthTokenType.Bot;
IsBotToken = _settingsService.LastToken.Kind == AuthTokenKind.Bot;
TokenValue = _settingsService.LastToken.Value;
}
@@ -157,7 +157,7 @@ namespace DiscordChatExporter.Gui.ViewModels
return;
var token = new AuthToken(
IsBotToken ? AuthTokenType.Bot : AuthTokenType.User,
IsBotToken ? AuthTokenKind.Bot : AuthTokenKind.User,
tokenValue
);

View File

@@ -18,6 +18,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DiscordChatExporter.Cli", "
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DiscordChatExporter.Core", "DiscordChatExporter.Core\DiscordChatExporter.Core.csproj", "{E19980B9-2B84-4257-A517-540FF1E3FCDD}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DiscordChatExporter.Cli.Tests", "DiscordChatExporter.Cli.Tests\DiscordChatExporter.Cli.Tests.csproj", "{C5064B8B-692E-4515-BA55-A9BE392EE540}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@@ -36,6 +38,10 @@ Global
{E19980B9-2B84-4257-A517-540FF1E3FCDD}.Debug|Any CPU.Build.0 = Debug|Any CPU
{E19980B9-2B84-4257-A517-540FF1E3FCDD}.Release|Any CPU.ActiveCfg = Release|Any CPU
{E19980B9-2B84-4257-A517-540FF1E3FCDD}.Release|Any CPU.Build.0 = Release|Any CPU
{C5064B8B-692E-4515-BA55-A9BE392EE540}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C5064B8B-692E-4515-BA55-A9BE392EE540}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C5064B8B-692E-4515-BA55-A9BE392EE540}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C5064B8B-692E-4515-BA55-A9BE392EE540}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE

View File

@@ -1,8 +1,10 @@
# DiscordChatExporter
[![Build](https://github.com/Tyrrrz/DiscordChatExporter/workflows/CI/badge.svg?branch=master)](https://github.com/Tyrrrz/DiscordChatExporter/actions)
[![Coverage](https://codecov.io/gh/Tyrrrz/DiscordChatExporter/branch/master/graph/badge.svg)](https://codecov.io/gh/Tyrrrz/DiscordChatExporter)
[![Release](https://img.shields.io/github/release/Tyrrrz/DiscordChatExporter.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
[![Downloads](https://img.shields.io/github/downloads/Tyrrrz/DiscordChatExporter/total.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
[![Discord](https://img.shields.io/discord/869237470565392384?label=discord)](https://discord.gg/2SUWKFnHSm)
[![Donate](https://img.shields.io/badge/donate-$$$-purple.svg)](https://tyrrrz.me/donate)
**Project status: active**.
@@ -12,6 +14,8 @@ It works with direct messages, group messages, and server channels, and supports
**If you have questions or issues, please refer to the [wiki](https://github.com/Tyrrrz/DiscordChatExporter/wiki)**.
💬 **If you want to chat, join my [Discord server](https://discord.gg/2SUWKFnHSm)**.
## Download
This application comes in two flavors: graphical user interface (**GUI**) and command line interface (**CLI**).
@@ -31,7 +35,7 @@ The following table lists all available download options:
<td>
<ul>
<li>🟢 <b><a href="https://github.com/Tyrrrz/DiscordChatExporter/releases/latest">Stable release</a></b> (<code>DiscordChatExporter.zip</code>)</li>
<li>🟠 <a href="https://github.com/Tyrrrz/DiscordChatExporter/actions?query=workflow%3ACI">CI build</a> (<code>DiscordChatExporter.zip</code>)</li>
<li>🟠 <a href="https://github.com/Tyrrrz/DiscordChatExporter/actions/workflows/CI.yml">CI build</a> (<code>DiscordChatExporter.zip</code>)</li>
</ul>
</td>
<td>
@@ -45,7 +49,7 @@ The following table lists all available download options:
<td>
<ul>
<li>🟢 <b><a href="https://github.com/Tyrrrz/DiscordChatExporter/releases/latest">Stable release</a></b> (<code>DiscordChatExporter.CLI.zip</code>) ⚙️</li>
<li>🟠 <a href="https://github.com/Tyrrrz/DiscordChatExporter/actions?query=workflow%3ACI">CI build</a> (<code>DiscordChatExporter.CLI.zip</code>) ⚙️</li>
<li>🟠 <a href="https://github.com/Tyrrrz/DiscordChatExporter/actions/workflows/CI.yml">CI build</a> (<code>DiscordChatExporter.CLI.zip</code>) ⚙️</li>
<li>🐋 <a href="https://hub.docker.com/r/tyrrrz/discordchatexporter">Docker</a> (<code>tyrrrz/discordchatexporter</code>)</li>
<li>📦 <a href="https://aur.archlinux.org/packages/discord-chat-exporter-cli">AUR</a> (<code>discord-chat-exporter-cli</code>) 🦄</li>
</ul>
@@ -75,7 +79,7 @@ The following table lists all available download options:
- Support for both user and bot tokens
- Support for Discord's dialect of markdown
- Support for Discord's message filter syntax
- Support for attachments, embeds, emojis, and other rich media features
- Support for attachments, embeds, emoji, and other rich media features
- Multiple output formats: HTML (dark/light), TXT, CSV, JSON
- File partitioning, date ranges, and other export options
- Exports messages on the fly without buffering in-memory