Compare commits

...

26 Commits

Author SHA1 Message Date
Tyrrrz 7d458cd3fd Update version 2023-01-08 18:42:35 +02:00
Oleksii Holub 57eb32d9c0 Update dockerfile 2023-01-05 14:20:49 +02:00
Tyrrrz 506bc0176b Fix incorrect replace order 2023-01-03 10:44:31 +02:00
Kuba_Z2 67b31c5f68 Remove regex check in Snowflake.TryParse (#980) 2023-01-03 10:41:59 +02:00
Tyrrrz d9c06bacda Trim dots at the end of filenames on Windows
Closes #977
2023-01-03 10:34:53 +02:00
Velithris db45b49af7 Use Alpine as the base image in docker, publish a self-contained app (#974) 2022-12-31 08:37:18 +02:00
Tyrrrz fc07f8da6c Fix the build badge in the readme 2022-12-16 20:54:39 +02:00
Tyrrrz b138908eb3 Re-add rate limit margins 2022-12-15 20:40:28 +02:00
Tyrrrz 2c4f812d4f Add link to Discord rate limit docs in comments 2022-12-15 20:19:12 +02:00
Tyrrrz 324d6bab20 Don't add margin for rate limit delays 2022-12-15 20:18:32 +02:00
Tyrrrz e5d5f1c85f Cleanup 2022-12-15 20:08:14 +02:00
Tyrrrz 49c4b12512 Smarter rate limit handling 2022-12-15 19:55:50 +02:00
Tyrrrz 57e2dc9313 Correct a comment 2022-12-13 03:15:28 +02:00
Oleksii Holub 07b72b9023 Update NuGet packages 2022-12-08 22:19:19 +02:00
Oleksii Holub f3830247fe Make use of the required keyword 2022-12-08 03:20:28 +02:00
Tyrrrz a8f89ec292 Remove unneeded project props 2022-12-08 01:49:32 +02:00
Tyrrrz 0bdbd44492 Update NuGet packages 2022-12-08 01:44:47 +02:00
Tyrrrz db23971b08 Use net7.0 2022-12-08 01:44:26 +02:00
m1337v 90bf4975b9 Update Readme.md to link to the latest .NET Runtime version for MacOS (#965) 2022-11-16 12:09:40 +02:00
Tyrrrz c2f9868746 More checks 2022-11-15 12:43:07 +02:00
Oleksii Holub ec840e76ae Simplify anti-russian checks 2022-11-15 01:36:23 +02:00
Oleksii Holub b672c30071 Pass cancellation token to markdown visitor 2022-11-03 19:13:50 +02:00
Oleksii Holub e752269467 Migrate from MiniRazor to RazorBlade 2022-11-03 18:56:45 +02:00
Tyrrrz 5b1b720503 Update Ukraine support links 2022-11-01 22:35:40 +02:00
Oleksii Holub 3dc3e3fa01 Update NuGet packages 2022-11-01 18:43:08 +02:00
Oleksii Holub 001ea7e495 Add region-based sanctions 2022-11-01 18:42:42 +02:00
42 changed files with 600 additions and 344 deletions
+2 -2
View File
@@ -18,7 +18,7 @@ jobs:
- name: Install .NET
uses: actions/setup-dotnet@v2
with:
dotnet-version: 6.0.x
dotnet-version: 7.0.x
- name: Run tests
# Tests need access to secrets, so we can't run them against PRs because of limited trust
@@ -49,7 +49,7 @@ jobs:
- name: Install .NET
uses: actions/setup-dotnet@v2
with:
dotnet-version: 6.0.x
dotnet-version: 7.0.x
- name: Publish (CLI)
run: dotnet publish DiscordChatExporter.Cli/ -o DiscordChatExporter.Cli/bin/Publish/ --configuration Release
+8
View File
@@ -1,3 +1,11 @@
### v2.37 (08-Jan-2023)
- Switched from .NET 6.0 to .NET 7.0. If running on Windows, the application should update all required prerequisites automatically. Alternatively, you can download the latest version of the runtime for your system [here](https://dotnet.microsoft.com/download/dotnet/7.0).
- Updated rate limit handling to adjust automatically based on the response headers. This may result in exports taking longer than before on bot accounts, but it makes it less likely for Discord to throttle you. Exports with user accounts shouldn't be affected too much by this change.
- Optimized snowflake parsing by removing unnecessary checks. (Thanks [@Kuba_Z2](https://github.com/KubaZ2))
- [Docker] Optimized the Docker image size by using the Alpine base image and publishing a self-contained distribution. (Thanks [@Velithris](https://github.com/Zireael-N))
- Fixed an issue where exporting a channel whose name ends with a dot would fail on Windows with default output path. The dots are now trimmed in file and directory names to match the behavior of Windows Explorer.
### v2.36.4 (29-Oct-2022)
- Changed all mentions of "media" in the context of "download media" or "reuse media" to "assets". CLI options will retain their existing names for backwards compatibility.
+2 -10
View File
@@ -1,8 +1,8 @@
<Project>
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<Version>2.36.4</Version>
<TargetFramework>net7.0</TargetFramework>
<Version>2.37</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Oleksii Holub</Copyright>
<LangVersion>preview</LangVersion>
@@ -10,12 +10,4 @@
<WarningsAsErrors>nullable</WarningsAsErrors>
</PropertyGroup>
<!--
Even though the project builds against .NET 6, some dependencies
apparently rely on a specific version of the runtime.
-->
<PropertyGroup>
<RuntimeFrameworkVersion>6.0.9</RuntimeFrameworkVersion>
</PropertyGroup>
</Project>
@@ -15,11 +15,11 @@
<PackageReference Include="FluentAssertions" Version="6.8.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="2.0.1" PrivateAssets="all" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.4.0" />
<PackageReference Include="System.Reactive" Version="5.0.0" />
<PackageReference Include="xunit" Version="2.4.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" PrivateAssets="all" />
<PackageReference Include="coverlet.collector" Version="3.1.2" PrivateAssets="all" />
<PackageReference Include="coverlet.collector" Version="3.2.0" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
+31 -9
View File
@@ -1,5 +1,7 @@
# Build
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
# -- Build
FROM mcr.microsoft.com/dotnet/sdk:7.0-alpine AS build
WORKDIR /build
COPY favicon.ico ./
COPY NuGet.config ./
@@ -7,16 +9,36 @@ COPY Directory.Build.props ./
COPY DiscordChatExporter.Core ./DiscordChatExporter.Core
COPY DiscordChatExporter.Cli ./DiscordChatExporter.Cli
RUN dotnet publish DiscordChatExporter.Cli --configuration Release --output ./publish
RUN dotnet publish DiscordChatExporter.Cli \
--self-contained \
--use-current-runtime \
--configuration Release \
--output ./publish
# Run
FROM mcr.microsoft.com/dotnet/runtime:6.0 AS run
# -- Run
FROM mcr.microsoft.com/dotnet/runtime-deps:7.0-alpine
# Alpine dotnet image doesn't include timezone data, which is needed
# for certain date/time operations.
RUN apk add --no-cache tzdata
# Create a non-root user to run the app, so that the output files
# can be accessed by the host.
# https://github.com/Tyrrrz/DiscordChatExporter/issues/851
RUN adduser \
--disabled-password \
--no-create-home \
dce
RUN useradd dce
USER dce
COPY --from=build ./publish ./
COPY --from=build /build/publish /opt/dce
WORKDIR ./out
# Need to keep this as /out for backwards compatibility with documentation.
# A lot of people have this directory mounted in their scripts files, so
# changing it would break existing workflows.
WORKDIR /out
ENTRYPOINT ["dotnet", "../DiscordChatExporter.Cli.dll"]
# Add the app directory to PATH so that it's easier to debug using a shell
ENV PATH="$PATH:/opt/dce"
ENTRYPOINT ["DiscordChatExporter.Cli"]
@@ -31,7 +31,7 @@ public abstract class ExportCommandBase : TokenCommandBase
public string OutputPath
{
get => _outputPath;
// Handle ~/ in paths on *nix systems
// Handle ~/ in paths on Unix systems
// https://github.com/Tyrrrz/DiscordChatExporter/pull/903
init => _outputPath = Path.GetFullPath(value);
}
@@ -236,8 +236,8 @@ public abstract class ExportCommandBase : TokenCommandBase
{
// War in Ukraine message
console.Output.WriteLine("========================================================================");
console.Output.WriteLine("|| Ukraine is at war! Support my country in its fight for freedom~ ||");
console.Output.WriteLine("|| Learn more on my website: https://tyrrrz.me ||");
console.Output.WriteLine("|| Ukraine is at war! Support my country in its fight for freedom ||");
console.Output.WriteLine("|| Learn more: https://tyrrrz.me/ukraine ||");
console.Output.WriteLine("========================================================================");
console.Output.WriteLine("");
@@ -12,11 +12,10 @@ public abstract class TokenCommandBase : ICommand
[CommandOption(
"token",
't',
IsRequired = true,
EnvironmentVariable = "DISCORD_TOKEN",
Description = "Authentication token."
)]
public string Token { get; init; } = "";
public required string Token { get; init; }
[Obsolete("This option doesn't do anything. Kept for backwards compatibility.")]
[CommandOption(
@@ -15,10 +15,9 @@ public class ExportChannelsCommand : ExportCommandBase
[CommandOption(
"channel",
'c',
IsRequired = true,
Description = "Channel ID(s)."
)]
public IReadOnlyList<Snowflake> ChannelIds { get; init; } = Array.Empty<Snowflake>();
public required IReadOnlyList<Snowflake> ChannelIds { get; init; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -15,10 +15,9 @@ public class ExportGuildCommand : ExportCommandBase
[CommandOption(
"guild",
'g',
IsRequired = true,
Description = "Guild ID."
)]
public Snowflake GuildId { get; init; }
public required Snowflake GuildId { get; init; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -16,10 +16,9 @@ public class GetChannelsCommand : TokenCommandBase
[CommandOption(
"guild",
'g',
IsRequired = true,
Description = "Guild ID."
)]
public Snowflake GuildId { get; init; }
public required Snowflake GuildId { get; init; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -6,10 +6,10 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliFx" Version="2.3.0" />
<PackageReference Include="CliFx" Version="2.3.1" />
<PackageReference Include="Spectre.Console" Version="0.45.0" />
<PackageReference Include="Gress" Version="2.0.1" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.3.1" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.4.0" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
+51
View File
@@ -0,0 +1,51 @@
using System;
using System.Globalization;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using Microsoft.Win32;
namespace DiscordChatExporter.Cli;
public static class Sanctions
{
[ModuleInitializer]
internal static void Verify()
{
var isSkipped = string.Equals(
Environment.GetEnvironmentVariable("RUSNI"),
"PYZDA",
StringComparison.OrdinalIgnoreCase
);
if (isSkipped)
return;
var locale = CultureInfo.CurrentCulture.Name;
var region =
RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
? Registry.CurrentUser
.OpenSubKey(@"Control Panel\International\Geo", false)?
.GetValue("Name") as string
: null;
var isSanctioned =
locale.EndsWith("-ru", StringComparison.OrdinalIgnoreCase) ||
locale.EndsWith("-by", StringComparison.OrdinalIgnoreCase) ||
string.Equals(region, "ru", StringComparison.OrdinalIgnoreCase) ||
string.Equals(region, "by", StringComparison.OrdinalIgnoreCase);
if (!isSanctioned)
return;
Console.ForegroundColor = ConsoleColor.Red;
Console.Error.WriteLine(
"You cannot use this software on the territory of a terrorist state. " +
"Set the environment variable `RUSNI=PYZDA` if you wish to override this check."
);
Console.ResetColor();
Environment.Exit(0xFACC);
}
}
@@ -1,5 +1,6 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
namespace DiscordChatExporter.Core.Discord.Data.Common;
@@ -39,7 +40,8 @@ public readonly partial record struct FileSize(long TotalBytes)
}
[ExcludeFromCodeCoverage]
public override string ToString() => $"{GetLargestWholeNumberValue():0.##} {GetLargestWholeNumberSymbol()}";
public override string ToString() =>
string.Create(CultureInfo.InvariantCulture, $"{GetLargestWholeNumberValue():0.##} {GetLargestWholeNumberSymbol()}");
}
public partial record struct FileSize
@@ -1,5 +1,6 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Net;
using System.Net.Http;
@@ -35,7 +36,7 @@ public class DiscordClient
{
using var request = new HttpRequestMessage(HttpMethod.Get, new Uri(_baseUri, url));
// Don't validate because token can have invalid characters
// Don't validate because the token can have special characters
// https://github.com/Tyrrrz/DiscordChatExporter/issues/828
request.Headers.TryAddWithoutValidation(
"Authorization",
@@ -44,11 +45,43 @@ public class DiscordClient
: _token
);
return await Http.Client.SendAsync(
var response = await Http.Client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead,
innerCancellationToken
);
// If this was the last request available before hitting the rate limit,
// wait out the reset time so that future requests can succeed.
// This may add an unnecessary delay in case the user doesn't intend to
// make any more requests, but implementing a smarter solution would
// require properly keeping track of Discord's global/per-route/per-resource
// rate limits and that's just way too much effort.
// https://discord.com/developers/docs/topics/rate-limits
var remainingRequestCount = response
.Headers
.TryGetValue("X-RateLimit-Remaining")?
.Pipe(s => int.Parse(s, CultureInfo.InvariantCulture));
var resetAfterDelay = response
.Headers
.TryGetValue("X-RateLimit-Reset-After")?
.Pipe(s => TimeSpan.FromSeconds(double.Parse(s, CultureInfo.InvariantCulture)));
if (remainingRequestCount <= 0 && resetAfterDelay is not null)
{
var delay =
// Adding a small buffer to the reset time reduces the chance of getting
// rate limited again, because it allows for more requests to be released.
(resetAfterDelay.Value + TimeSpan.FromSeconds(1))
// Sometimes Discord returns an absurdly high value for the reset time, which
// is not actually enforced by the server. So we cap it at a reasonable value.
.Clamp(TimeSpan.Zero, TimeSpan.FromSeconds(60));
await Task.Delay(delay, innerCancellationToken);
}
return response;
}, cancellationToken);
}
@@ -1,7 +1,6 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord;
@@ -29,7 +28,7 @@ public partial record struct Snowflake
return null;
// As number
if (Regex.IsMatch(str, @"^\d+$") && ulong.TryParse(str, NumberStyles.Number, formatProvider, out var value))
if (ulong.TryParse(str, NumberStyles.None, formatProvider, out var value))
{
return new Snowflake(value);
}
@@ -7,14 +7,14 @@
<ItemGroup>
<PackageReference Include="Gress" Version="2.0.1" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="MiniRazor.CodeGen" Version="2.2.2" />
<PackageReference Include="Polly" Version="7.2.3" />
<PackageReference Include="RazorBlade" Version="0.4.1" />
<PackageReference Include="Superpower" Version="3.0.0" />
<PackageReference Include="WebMarkupMin.Core" Version="2.12.0" />
<PackageReference Include="WebMarkupMin.Core" Version="2.13.0" />
</ItemGroup>
<ItemGroup>
<AdditionalFiles Include="Exporting\Writers\Html\*.cshtml" IsRazorTemplate="true" />
<RazorBlade Include="Exporting\Writers\Html\*.cshtml" />
</ItemGroup>
</Project>
@@ -40,45 +40,6 @@ public partial record ExportRequest(
public partial record ExportRequest
{
private static string GetOutputBaseFilePath(
Guild guild,
Channel channel,
string outputPath,
ExportFormat format,
Snowflake? after = null,
Snowflake? before = null)
{
// Formats path
outputPath = Regex.Replace(outputPath, "%.", m =>
PathEx.EscapeFileName(m.Value switch
{
"%g" => guild.Id.ToString(),
"%G" => guild.Name,
"%t" => channel.Category.Id.ToString(),
"%T" => channel.Category.Name,
"%c" => channel.Id.ToString(),
"%C" => channel.Name,
"%p" => channel.Position?.ToString() ?? "0",
"%P" => channel.Category.Position?.ToString() ?? "0",
"%a" => after?.ToDate().ToString("yyyy-MM-dd") ?? "",
"%b" => before?.ToDate().ToString("yyyy-MM-dd") ?? "",
"%d" => DateTimeOffset.Now.ToString("yyyy-MM-dd"),
"%%" => "%",
_ => m.Value
})
);
// Output is a directory
if (Directory.Exists(outputPath) || string.IsNullOrWhiteSpace(Path.GetExtension(outputPath)))
{
var fileName = GetDefaultOutputFileName(guild, channel, format, after, before);
return Path.Combine(outputPath, fileName);
}
// Output is a file
return outputPath;
}
public static string GetDefaultOutputFileName(
Guild guild,
Channel channel,
@@ -120,4 +81,45 @@ public partial record ExportRequest
return PathEx.EscapeFileName(buffer.ToString());
}
private static string GetOutputBaseFilePath(
Guild guild,
Channel channel,
string outputPath,
ExportFormat format,
Snowflake? after = null,
Snowflake? before = null)
{
// Format path
var actualOutputPath = Regex.Replace(
outputPath,
"%.",
m => PathEx.EscapeFileName(m.Value switch
{
"%g" => guild.Id.ToString(),
"%G" => guild.Name,
"%t" => channel.Category.Id.ToString(),
"%T" => channel.Category.Name,
"%c" => channel.Id.ToString(),
"%C" => channel.Name,
"%p" => channel.Position?.ToString() ?? "0",
"%P" => channel.Category.Position?.ToString() ?? "0",
"%a" => after?.ToDate().ToString("yyyy-MM-dd") ?? "",
"%b" => before?.ToDate().ToString("yyyy-MM-dd") ?? "",
"%d" => DateTimeOffset.Now.ToString("yyyy-MM-dd"),
"%%" => "%",
_ => m.Value
})
);
// Output is a directory
if (Directory.Exists(actualOutputPath) || string.IsNullOrWhiteSpace(Path.GetExtension(actualOutputPath)))
{
var fileName = GetDefaultOutputFileName(guild, channel, format, after, before);
return Path.Combine(actualOutputPath, fileName);
}
// Output is a file
return actualOutputPath;
}
}
@@ -19,8 +19,10 @@ internal partial class CsvMessageWriter : MessageWriter
_writer = new StreamWriter(stream);
}
private ValueTask<string> FormatMarkdownAsync(string? markdown) =>
PlainTextMarkdownVisitor.FormatAsync(Context, markdown ?? "");
private ValueTask<string> FormatMarkdownAsync(
string markdown,
CancellationToken cancellationToken = default) =>
PlainTextMarkdownVisitor.FormatAsync(Context, markdown, cancellationToken);
public override async ValueTask WritePreambleAsync(CancellationToken cancellationToken = default) =>
await _writer.WriteLineAsync("AuthorID,Author,Date,Content,Attachments,Reactions");
@@ -84,7 +86,7 @@ internal partial class CsvMessageWriter : MessageWriter
await _writer.WriteAsync(',');
// Message content
await _writer.WriteAsync(CsvEncode(await FormatMarkdownAsync(message.Content)));
await _writer.WriteAsync(CsvEncode(await FormatMarkdownAsync(message.Content, cancellationToken)));
await _writer.WriteAsync(',');
// Attachments
@@ -1,43 +1,52 @@
@using System
@using System.Collections.Generic
@using System.Linq
@using System.Threading
@using System.Threading.Tasks
@using DiscordChatExporter.Core.Discord.Data
@using DiscordChatExporter.Core.Discord.Data.Embeds
@using DiscordChatExporter.Core.Exporting.Writers.Html;
@using DiscordChatExporter.Core.Exporting
@using DiscordChatExporter.Core.Exporting.Writers.Html
@using DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
@using DiscordChatExporter.Core.Utils.Extensions
@namespace DiscordChatExporter.Core.Exporting.Writers.Html
@inherits MiniRazor.TemplateBase<MessageGroupTemplateContext>
@inherits RazorBlade.HtmlTemplate
@functions {
public required ExportContext ExportContext { get; init; }
public required IReadOnlyList<Message> Messages { get; init; }
}
@{
ValueTask<string> ResolveAssetUrlAsync(string url) =>
Model.ExportContext.ResolveAssetUrlAsync(url);
ExportContext.ResolveAssetUrlAsync(url, CancellationToken);
string FormatDate(DateTimeOffset date) =>
Model.ExportContext.FormatDate(date);
ExportContext.FormatDate(date);
ValueTask<string> FormatMarkdownAsync(string markdown) =>
Model.FormatMarkdownAsync(markdown);
HtmlMarkdownVisitor.FormatAsync(ExportContext, markdown, true, CancellationToken);
ValueTask<string> FormatEmbedMarkdownAsync(string markdown) =>
Model.FormatMarkdownAsync(markdown, false);
HtmlMarkdownVisitor.FormatAsync(ExportContext, markdown, false, CancellationToken);
var firstMessage = Model.Messages.First();
var firstMessage = Messages.First();
var userMember = Model.ExportContext.TryGetMember(firstMessage.Author.Id);
var userMember = ExportContext.TryGetMember(firstMessage.Author.Id);
var userColor = Model.ExportContext.TryGetUserColor(firstMessage.Author.Id);
var userColor = ExportContext.TryGetUserColor(firstMessage.Author.Id);
var userNick = firstMessage.Author.IsBot
? firstMessage.Author.Name
: userMember?.Nick ?? firstMessage.Author.Name;
var referencedUserMember = firstMessage.ReferencedMessage is not null
? Model.ExportContext.TryGetMember(firstMessage.ReferencedMessage.Author.Id)
? ExportContext.TryGetMember(firstMessage.ReferencedMessage.Author.Id)
: null;
var referencedUserColor = firstMessage.ReferencedMessage is not null
? Model.ExportContext.TryGetUserColor(firstMessage.ReferencedMessage.Author.Id)
? ExportContext.TryGetUserColor(firstMessage.ReferencedMessage.Author.Id)
: null;
var referencedUserNick = firstMessage.ReferencedMessage is not null
@@ -48,7 +57,7 @@
}
<div class="chatlog__message-group">
@foreach (var (message, i) in Model.Messages.WithIndex())
@foreach (var (message, i) in Messages.WithIndex())
{
var isFirst = i == 0;
@@ -59,8 +68,8 @@
{
// System notifications are grouped even if the message author is different.
// That's why we have to update the user values with the author of the current message.
userMember = Model.ExportContext.TryGetMember(message.Author.Id);
userColor = Model.ExportContext.TryGetUserColor(message.Author.Id);
userMember = ExportContext.TryGetMember(message.Author.Id);
userColor = ExportContext.TryGetUserColor(message.Author.Id);
userNick = message.Author.IsBot
? message.Author.Name
: userMember?.Nick ?? message.Author.Name;
@@ -129,7 +138,7 @@
<span class="chatlog__reference-link" onclick="scrollToMessage(event, '@message.ReferencedMessage.Id')">
@if (!string.IsNullOrWhiteSpace(message.ReferencedMessage.Content) && !message.ReferencedMessage.IsContentHidden())
{
<!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(message.ReferencedMessage.Content))<!--/wmm:ignore-->
<!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(message.ReferencedMessage.Content))<!--/wmm:ignore-->
}
else if (message.ReferencedMessage.Attachments.Any() || message.ReferencedMessage.Embeds.Any())
{
@@ -180,7 +189,7 @@
@{/* Text */}
@if (!string.IsNullOrWhiteSpace(message.Content) && !message.IsContentHidden())
{
<span class="chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatMarkdownAsync(message.Content))<!--/wmm:ignore--></span>
<span class="chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatMarkdownAsync(message.Content))<!--/wmm:ignore--></span>
}
@{/* Edited timestamp */}
@@ -300,12 +309,12 @@
@if (!string.IsNullOrWhiteSpace(embed.Url))
{
<a class="chatlog__embed-title-link" href="@embed.Url">
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
</a>
}
else
{
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
}
</div>
}
@@ -386,12 +395,12 @@
@if (!string.IsNullOrWhiteSpace(embed.Url))
{
<a class="chatlog__embed-title-link" href="@embed.Url">
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
</a>
}
else
{
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(embed.Title))<!--/wmm:ignore--></div>
}
</div>
}
@@ -400,7 +409,7 @@
@if (!string.IsNullOrWhiteSpace(embed.Description))
{
<div class="chatlog__embed-description">
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(embed.Description))<!--/wmm:ignore--></div>
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(embed.Description))<!--/wmm:ignore--></div>
</div>
}
@@ -414,14 +423,14 @@
@if (!string.IsNullOrWhiteSpace(field.Name))
{
<div class="chatlog__embed-field-name">
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(field.Name))<!--/wmm:ignore--></div>
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(field.Name))<!--/wmm:ignore--></div>
</div>
}
@if (!string.IsNullOrWhiteSpace(field.Value))
{
<div class="chatlog__embed-field-value">
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Raw(await FormatEmbedMarkdownAsync(field.Value))<!--/wmm:ignore--></div>
<div class="chatlog__markdown chatlog__markdown-preserve"><!--wmm:ignore-->@Html.Raw(await FormatEmbedMarkdownAsync(field.Value))<!--/wmm:ignore--></div>
</div>
}
</div>
@@ -1,22 +0,0 @@
using System.Collections.Generic;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors;
namespace DiscordChatExporter.Core.Exporting.Writers.Html;
internal class MessageGroupTemplateContext
{
public ExportContext ExportContext { get; }
public IReadOnlyList<Message> Messages { get; }
public MessageGroupTemplateContext(ExportContext exportContext, IReadOnlyList<Message> messages)
{
ExportContext = exportContext;
Messages = messages;
}
public ValueTask<string> FormatMarkdownAsync(string? markdown, bool isJumboAllowed = true) =>
HtmlMarkdownVisitor.FormatAsync(ExportContext, markdown ?? "", isJumboAllowed);
}
@@ -1,15 +1,23 @@
@using DiscordChatExporter.Core.Exporting.Writers.Html;
@using System.Threading
@using DiscordChatExporter.Core.Exporting
@namespace DiscordChatExporter.Core.Exporting.Writers.Html
@inherits MiniRazor.TemplateBase<PostambleTemplateContext>
@inherits RazorBlade.HtmlTemplate
@{/* Close elements opened by preamble */}
@functions {
public required ExportContext ExportContext { get; init; }
public required long MessagesWritten { get; init; }
}
@{
/* Close elements opened by preamble */
}
<!--wmm:ignore-->
</div>
<!--/wmm:ignore-->
<div class="postamble">
<div class="postamble__entry">Exported @Model.MessagesWritten.ToString("N0") message(s)</div>
<div class="postamble__entry">Exported @MessagesWritten.ToString("N0") message(s)</div>
</div>
</body>
@@ -1,14 +0,0 @@
namespace DiscordChatExporter.Core.Exporting.Writers.Html;
internal class PostambleTemplateContext
{
public ExportContext ExportContext { get; }
public long MessagesWritten { get; }
public PostambleTemplateContext(ExportContext exportContext, long messagesWritten)
{
ExportContext = exportContext;
MessagesWritten = messagesWritten;
}
}
@@ -1,13 +1,20 @@
@using System
@using System.Threading
@using System.Threading.Tasks
@using DiscordChatExporter.Core.Exporting.Writers.Html;
@using DiscordChatExporter.Core.Exporting
@using DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
@namespace DiscordChatExporter.Core.Exporting.Writers.Html
@inherits MiniRazor.TemplateBase<PreambleTemplateContext>
@inherits RazorBlade.HtmlTemplate
@functions {
public required ExportContext ExportContext { get; init; }
public required string ThemeName { get; init; }
}
@{
string Themed(string darkVariant, string lightVariant) =>
string.Equals(Model.ThemeName, "Dark", StringComparison.OrdinalIgnoreCase)
string.Equals(ThemeName, "Dark", StringComparison.OrdinalIgnoreCase)
? darkVariant
: lightVariant;
@@ -15,20 +22,20 @@
$"https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-{weight}.woff";
ValueTask<string> ResolveAssetUrlAsync(string url) =>
Model.ExportContext.ResolveAssetUrlAsync(url, CancellationToken);
ExportContext.ResolveAssetUrlAsync(url, CancellationToken);
string FormatDate(DateTimeOffset date) =>
Model.ExportContext.FormatDate(date);
ExportContext.FormatDate(date);
ValueTask<string> FormatMarkdownAsync(string markdown) =>
Model.FormatMarkdownAsync(markdown);
HtmlMarkdownVisitor.FormatAsync(ExportContext, markdown, true, CancellationToken);
}
<!DOCTYPE html>
<html lang="en">
<head>
<title>@Model.ExportContext.Request.Guild.Name - @Model.ExportContext.Request.Channel.Name</title>
<title>@ExportContext.Request.Guild.Name - @ExportContext.Request.Channel.Name</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width">
@@ -755,7 +762,7 @@
</style>
@{/* Syntax highlighting */}
<link rel="stylesheet" href="@await ResolveAssetUrlAsync($"https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/styles/solarized-{Model.ThemeName.ToLowerInvariant()}.min.css")">
<link rel="stylesheet" href="@await ResolveAssetUrlAsync($"https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/styles/solarized-{ThemeName.ToLowerInvariant()}.min.css")">
<script src="@await ResolveAssetUrlAsync("https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/highlight.min.js")"></script>
<script>
document.addEventListener('DOMContentLoaded', () => {
@@ -846,31 +853,31 @@
<div class="preamble">
<div class="preamble__guild-icon-container">
<img class="preamble__guild-icon" src="@await ResolveAssetUrlAsync(Model.ExportContext.Request.Guild.IconUrl)" alt="Guild icon" loading="lazy">
<img class="preamble__guild-icon" src="@await ResolveAssetUrlAsync(ExportContext.Request.Guild.IconUrl)" alt="Guild icon" loading="lazy">
</div>
<div class="preamble__entries-container">
<div class="preamble__entry">@Model.ExportContext.Request.Guild.Name</div>
<div class="preamble__entry">@Model.ExportContext.Request.Channel.Category.Name / @Model.ExportContext.Request.Channel.Name</div>
<div class="preamble__entry">@ExportContext.Request.Guild.Name</div>
<div class="preamble__entry">@ExportContext.Request.Channel.Category.Name / @ExportContext.Request.Channel.Name</div>
@if (!string.IsNullOrWhiteSpace(Model.ExportContext.Request.Channel.Topic))
@if (!string.IsNullOrWhiteSpace(ExportContext.Request.Channel.Topic))
{
<div class="preamble__entry preamble__entry--small">@Raw(await FormatMarkdownAsync(Model.ExportContext.Request.Channel.Topic))</div>
<div class="preamble__entry preamble__entry--small">@Html.Raw(await FormatMarkdownAsync(ExportContext.Request.Channel.Topic))</div>
}
@if (Model.ExportContext.Request.After is not null || Model.ExportContext.Request.Before is not null)
@if (ExportContext.Request.After is not null || ExportContext.Request.Before is not null)
{
<div class="preamble__entry preamble__entry--small">
@if (Model.ExportContext.Request.After is not null && Model.ExportContext.Request.Before is not null)
@if (ExportContext.Request.After is not null && ExportContext.Request.Before is not null)
{
@($"Between {FormatDate(Model.ExportContext.Request.After.Value.ToDate())} and {FormatDate(Model.ExportContext.Request.Before.Value.ToDate())}")
@($"Between {FormatDate(ExportContext.Request.After.Value.ToDate())} and {FormatDate(ExportContext.Request.Before.Value.ToDate())}")
}
else if (Model.ExportContext.Request.After is not null)
else if (ExportContext.Request.After is not null)
{
@($"After {FormatDate(Model.ExportContext.Request.After.Value.ToDate())}")
@($"After {FormatDate(ExportContext.Request.After.Value.ToDate())}")
}
else if (Model.ExportContext.Request.Before is not null)
else if (ExportContext.Request.Before is not null)
{
@($"Before {FormatDate(Model.ExportContext.Request.Before.Value.ToDate())}")
@($"Before {FormatDate(ExportContext.Request.Before.Value.ToDate())}")
}
</div>
}
@@ -1,20 +0,0 @@
using System.Threading.Tasks;
using DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors;
namespace DiscordChatExporter.Core.Exporting.Writers.Html;
internal class PreambleTemplateContext
{
public ExportContext ExportContext { get; }
public string ThemeName { get; }
public PreambleTemplateContext(ExportContext exportContext, string themeName)
{
ExportContext = exportContext;
ThemeName = themeName;
}
public ValueTask<string> FormatMarkdownAsync(string? markdown, bool isJumboAllowed = true) =>
HtmlMarkdownVisitor.FormatAsync(ExportContext, markdown ?? "", isJumboAllowed);
}
@@ -71,15 +71,16 @@ internal class HtmlMessageWriter : MessageWriter
.Minify(html, false)
.MinifiedContent;
public override async ValueTask WritePreambleAsync(CancellationToken cancellationToken = default)
public override async ValueTask WritePreambleAsync(
CancellationToken cancellationToken = default)
{
var templateContext = new PreambleTemplateContext(Context, _themeName);
// We are not writing directly to output because Razor
// does not actually do asynchronous writes to stream.
await _writer.WriteLineAsync(
Minify(
await PreambleTemplate.RenderAsync(templateContext, cancellationToken)
await new PreambleTemplate
{
ExportContext = Context,
ThemeName = _themeName
}.RenderAsync(cancellationToken)
)
);
}
@@ -88,13 +89,13 @@ internal class HtmlMessageWriter : MessageWriter
IReadOnlyList<Message> messages,
CancellationToken cancellationToken = default)
{
var templateContext = new MessageGroupTemplateContext(Context, messages);
// We are not writing directly to output because Razor
// does not actually do asynchronous writes to stream.
await _writer.WriteLineAsync(
Minify(
await MessageGroupTemplate.RenderAsync(templateContext, cancellationToken)
await new MessageGroupTemplate
{
ExportContext = Context,
Messages = messages
}.RenderAsync(cancellationToken)
)
);
}
@@ -126,13 +127,13 @@ internal class HtmlMessageWriter : MessageWriter
if (_messageGroup.Any())
await WriteMessageGroupAsync(_messageGroup, cancellationToken);
var templateContext = new PostambleTemplateContext(Context, MessagesWritten);
// We are not writing directly to output because Razor
// does not actually do asynchronous writes to stream.
await _writer.WriteLineAsync(
Minify(
await PostambleTemplate.RenderAsync(templateContext, cancellationToken)
await new PostambleTemplate
{
ExportContext = Context,
MessagesWritten = MessagesWritten
}.RenderAsync(cancellationToken)
)
);
}
@@ -29,8 +29,10 @@ internal class JsonMessageWriter : MessageWriter
});
}
private ValueTask<string> FormatMarkdownAsync(string? markdown) =>
PlainTextMarkdownVisitor.FormatAsync(Context, markdown ?? "");
private ValueTask<string> FormatMarkdownAsync(
string markdown,
CancellationToken cancellationToken = default) =>
PlainTextMarkdownVisitor.FormatAsync(Context, markdown, cancellationToken);
private async ValueTask WriteAttachmentAsync(
Attachment attachment,
@@ -115,8 +117,8 @@ internal class JsonMessageWriter : MessageWriter
{
_writer.WriteStartObject();
_writer.WriteString("name", await FormatMarkdownAsync(embedField.Name));
_writer.WriteString("value", await FormatMarkdownAsync(embedField.Value));
_writer.WriteString("name", await FormatMarkdownAsync(embedField.Name, cancellationToken));
_writer.WriteString("value", await FormatMarkdownAsync(embedField.Value, cancellationToken));
_writer.WriteBoolean("isInline", embedField.IsInline);
_writer.WriteEndObject();
@@ -129,10 +131,10 @@ internal class JsonMessageWriter : MessageWriter
{
_writer.WriteStartObject();
_writer.WriteString("title", await FormatMarkdownAsync(embed.Title));
_writer.WriteString("title", await FormatMarkdownAsync(embed.Title ?? "", cancellationToken));
_writer.WriteString("url", embed.Url);
_writer.WriteString("timestamp", embed.Timestamp);
_writer.WriteString("description", await FormatMarkdownAsync(embed.Description));
_writer.WriteString("description", await FormatMarkdownAsync(embed.Description ?? "", cancellationToken));
if (embed.Color is not null)
_writer.WriteString("color", embed.Color.Value.ToHex());
@@ -283,7 +285,7 @@ internal class JsonMessageWriter : MessageWriter
_writer.WriteBoolean("isPinned", message.IsPinned);
// Content
_writer.WriteString("content", await FormatMarkdownAsync(message.Content));
_writer.WriteString("content", await FormatMarkdownAsync(message.Content, cancellationToken));
// Author
_writer.WriteStartObject("author");
@@ -3,6 +3,7 @@ using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Markdown;
@@ -24,13 +25,17 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
_isJumbo = isJumbo;
}
protected override async ValueTask<MarkdownNode> VisitTextAsync(TextNode text)
protected override async ValueTask<MarkdownNode> VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default)
{
_buffer.Append(HtmlEncode(text.Text));
return await base.VisitTextAsync(text);
return await base.VisitTextAsync(text, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitFormattingAsync(FormattingNode formatting)
protected override async ValueTask<MarkdownNode> VisitFormattingAsync(
FormattingNode formatting,
CancellationToken cancellationToken = default)
{
var (openingTag, closingTag) = formatting.Kind switch
{
@@ -68,23 +73,27 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
};
_buffer.Append(openingTag);
var result = await base.VisitFormattingAsync(formatting);
var result = await base.VisitFormattingAsync(formatting, cancellationToken);
_buffer.Append(closingTag);
return result;
}
protected override async ValueTask<MarkdownNode> VisitInlineCodeBlockAsync(InlineCodeBlockNode inlineCodeBlock)
protected override async ValueTask<MarkdownNode> VisitInlineCodeBlockAsync(
InlineCodeBlockNode inlineCodeBlock,
CancellationToken cancellationToken = default)
{
_buffer
.Append("<code class=\"chatlog__markdown-pre chatlog__markdown-pre--inline\">")
.Append(HtmlEncode(inlineCodeBlock.Code))
.Append("</code>");
return await base.VisitInlineCodeBlockAsync(inlineCodeBlock);
return await base.VisitInlineCodeBlockAsync(inlineCodeBlock, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitMultiLineCodeBlockAsync(MultiLineCodeBlockNode multiLineCodeBlock)
protected override async ValueTask<MarkdownNode> VisitMultiLineCodeBlockAsync(
MultiLineCodeBlockNode multiLineCodeBlock,
CancellationToken cancellationToken = default)
{
var highlightCssClass = !string.IsNullOrWhiteSpace(multiLineCodeBlock.Language)
? $"language-{multiLineCodeBlock.Language}"
@@ -95,10 +104,12 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
.Append(HtmlEncode(multiLineCodeBlock.Code))
.Append("</code>");
return await base.VisitMultiLineCodeBlockAsync(multiLineCodeBlock);
return await base.VisitMultiLineCodeBlockAsync(multiLineCodeBlock, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitLinkAsync(LinkNode link)
protected override async ValueTask<MarkdownNode> VisitLinkAsync(
LinkNode link,
CancellationToken cancellationToken = default)
{
// Try to extract message ID if the link refers to a Discord message
var linkedMessageId = Regex.Match(
@@ -112,13 +123,15 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
: $"<a href=\"{HtmlEncode(link.Url)}\">"
);
var result = await base.VisitLinkAsync(link);
var result = await base.VisitLinkAsync(link, cancellationToken);
_buffer.Append("</a>");
return result;
}
protected override async ValueTask<MarkdownNode> VisitEmojiAsync(EmojiNode emoji)
protected override async ValueTask<MarkdownNode> VisitEmojiAsync(
EmojiNode emoji,
CancellationToken cancellationToken = default)
{
var emojiImageUrl = Emoji.GetImageUrl(emoji.Id, emoji.Name, emoji.IsAnimated);
var jumboClass = _isJumbo ? "chatlog__emoji--large" : "";
@@ -129,14 +142,16 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
$"class=\"chatlog__emoji {jumboClass}\" " +
$"alt=\"{emoji.Name}\" " +
$"title=\"{emoji.Code}\" " +
$"src=\"{await _context.ResolveAssetUrlAsync(emojiImageUrl)}\"" +
$"src=\"{await _context.ResolveAssetUrlAsync(emojiImageUrl, cancellationToken)}\"" +
$">"
);
return await base.VisitEmojiAsync(emoji);
return await base.VisitEmojiAsync(emoji, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitMentionAsync(MentionNode mention)
protected override async ValueTask<MarkdownNode> VisitMentionAsync(
MentionNode mention,
CancellationToken cancellationToken = default)
{
if (mention.Kind == MentionKind.Everyone)
{
@@ -191,10 +206,12 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
.Append("</span>");
}
return await base.VisitMentionAsync(mention);
return await base.VisitMentionAsync(mention, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitUnixTimestampAsync(UnixTimestampNode timestamp)
protected override async ValueTask<MarkdownNode> VisitUnixTimestampAsync(
UnixTimestampNode timestamp,
CancellationToken cancellationToken = default)
{
var dateString = timestamp.Date is not null
? _context.FormatDate(timestamp.Date.Value)
@@ -210,7 +227,7 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
.Append(HtmlEncode(dateString))
.Append("</span>");
return await base.VisitUnixTimestampAsync(timestamp);
return await base.VisitUnixTimestampAsync(timestamp, cancellationToken);
}
}
@@ -221,7 +238,8 @@ internal partial class HtmlMarkdownVisitor
public static async ValueTask<string> FormatAsync(
ExportContext context,
string markdown,
bool isJumboAllowed = true)
bool isJumboAllowed = true,
CancellationToken cancellationToken = default)
{
var nodes = MarkdownParser.Parse(markdown);
@@ -231,7 +249,8 @@ internal partial class HtmlMarkdownVisitor
var buffer = new StringBuilder();
await new HtmlMarkdownVisitor(context, buffer, isJumbo).VisitAsync(nodes);
await new HtmlMarkdownVisitor(context, buffer, isJumbo)
.VisitAsync(nodes, cancellationToken);
return buffer.ToString();
}
@@ -1,4 +1,5 @@
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Markdown.Parsing;
@@ -17,13 +18,17 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
_buffer = buffer;
}
protected override async ValueTask<MarkdownNode> VisitTextAsync(TextNode text)
protected override async ValueTask<MarkdownNode> VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default)
{
_buffer.Append(text.Text);
return await base.VisitTextAsync(text);
return await base.VisitTextAsync(text, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitEmojiAsync(EmojiNode emoji)
protected override async ValueTask<MarkdownNode> VisitEmojiAsync(
EmojiNode emoji,
CancellationToken cancellationToken = default)
{
_buffer.Append(
emoji.IsCustomEmoji
@@ -31,10 +36,12 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
: emoji.Name
);
return await base.VisitEmojiAsync(emoji);
return await base.VisitEmojiAsync(emoji, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitMentionAsync(MentionNode mention)
protected override async ValueTask<MarkdownNode> VisitMentionAsync(
MentionNode mention,
CancellationToken cancellationToken = default)
{
if (mention.Kind == MentionKind.Everyone)
{
@@ -70,10 +77,12 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
_buffer.Append($"@{name}");
}
return await base.VisitMentionAsync(mention);
return await base.VisitMentionAsync(mention, cancellationToken);
}
protected override async ValueTask<MarkdownNode> VisitUnixTimestampAsync(UnixTimestampNode timestamp)
protected override async ValueTask<MarkdownNode> VisitUnixTimestampAsync(
UnixTimestampNode timestamp,
CancellationToken cancellationToken = default)
{
_buffer.Append(
timestamp.Date is not null
@@ -81,18 +90,22 @@ internal partial class PlainTextMarkdownVisitor : MarkdownVisitor
: "Invalid date"
);
return await base.VisitUnixTimestampAsync(timestamp);
return await base.VisitUnixTimestampAsync(timestamp, cancellationToken);
}
}
internal partial class PlainTextMarkdownVisitor
{
public static async ValueTask<string> FormatAsync(ExportContext context, string markdown)
public static async ValueTask<string> FormatAsync(
ExportContext context,
string markdown,
CancellationToken cancellationToken = default)
{
var nodes = MarkdownParser.ParseMinimal(markdown);
var buffer = new StringBuilder();
await new PlainTextMarkdownVisitor(context, buffer).VisitAsync(nodes);
await new PlainTextMarkdownVisitor(context, buffer)
.VisitAsync(nodes, cancellationToken);
return buffer.ToString();
}
@@ -19,8 +19,10 @@ internal class PlainTextMessageWriter : MessageWriter
_writer = new StreamWriter(stream);
}
private ValueTask<string> FormatMarkdownAsync(string? markdown) =>
PlainTextMarkdownVisitor.FormatAsync(Context, markdown ?? "");
private ValueTask<string> FormatMarkdownAsync(
string markdown,
CancellationToken cancellationToken = default) =>
PlainTextMarkdownVisitor.FormatAsync(Context, markdown, cancellationToken);
private async ValueTask WriteMessageHeaderAsync(Message message)
{
@@ -48,7 +50,9 @@ internal class PlainTextMessageWriter : MessageWriter
{
cancellationToken.ThrowIfCancellationRequested();
await _writer.WriteLineAsync(await Context.ResolveAssetUrlAsync(attachment.Url, cancellationToken));
await _writer.WriteLineAsync(
await Context.ResolveAssetUrlAsync(attachment.Url, cancellationToken)
);
}
await _writer.WriteLineAsync();
@@ -65,24 +69,44 @@ internal class PlainTextMessageWriter : MessageWriter
await _writer.WriteLineAsync("{Embed}");
if (!string.IsNullOrWhiteSpace(embed.Author?.Name))
{
await _writer.WriteLineAsync(embed.Author.Name);
}
if (!string.IsNullOrWhiteSpace(embed.Url))
{
await _writer.WriteLineAsync(embed.Url);
}
if (!string.IsNullOrWhiteSpace(embed.Title))
await _writer.WriteLineAsync(await FormatMarkdownAsync(embed.Title));
{
await _writer.WriteLineAsync(
await FormatMarkdownAsync(embed.Title, cancellationToken)
);
}
if (!string.IsNullOrWhiteSpace(embed.Description))
await _writer.WriteLineAsync(await FormatMarkdownAsync(embed.Description));
{
await _writer.WriteLineAsync(
await FormatMarkdownAsync(embed.Description, cancellationToken)
);
}
foreach (var field in embed.Fields)
{
if (!string.IsNullOrWhiteSpace(field.Name))
await _writer.WriteLineAsync(await FormatMarkdownAsync(field.Name));
{
await _writer.WriteLineAsync(
await FormatMarkdownAsync(field.Name, cancellationToken)
);
}
if (!string.IsNullOrWhiteSpace(field.Value))
await _writer.WriteLineAsync(await FormatMarkdownAsync(field.Value));
{
await _writer.WriteLineAsync(
await FormatMarkdownAsync(field.Value, cancellationToken)
);
}
}
if (!string.IsNullOrWhiteSpace(embed.Thumbnail?.Url))
@@ -109,7 +133,9 @@ internal class PlainTextMessageWriter : MessageWriter
}
if (!string.IsNullOrWhiteSpace(embed.Footer?.Text))
{
await _writer.WriteLineAsync(embed.Footer.Text);
}
await _writer.WriteLineAsync();
}
@@ -152,7 +178,9 @@ internal class PlainTextMessageWriter : MessageWriter
await _writer.WriteAsync(reaction.Emoji.Name);
if (reaction.Count > 1)
{
await _writer.WriteAsync($" ({reaction.Count})");
}
await _writer.WriteAsync(' ');
}
@@ -167,13 +195,19 @@ internal class PlainTextMessageWriter : MessageWriter
await _writer.WriteLineAsync($"Channel: {Context.Request.Channel.Category.Name} / {Context.Request.Channel.Name}");
if (!string.IsNullOrWhiteSpace(Context.Request.Channel.Topic))
{
await _writer.WriteLineAsync($"Topic: {Context.Request.Channel.Topic}");
}
if (Context.Request.After is not null)
{
await _writer.WriteLineAsync($"After: {Context.FormatDate(Context.Request.After.Value.ToDate())}");
}
if (Context.Request.Before is not null)
{
await _writer.WriteLineAsync($"Before: {Context.FormatDate(Context.Request.Before.Value.ToDate())}");
}
await _writer.WriteLineAsync(new string('=', 62));
await _writer.WriteLineAsync();
@@ -190,7 +224,11 @@ internal class PlainTextMessageWriter : MessageWriter
// Content
if (!string.IsNullOrWhiteSpace(message.Content))
await _writer.WriteLineAsync(await FormatMarkdownAsync(message.Content));
{
await _writer.WriteLineAsync(
await FormatMarkdownAsync(message.Content, cancellationToken)
);
}
await _writer.WriteLineAsync();
@@ -1,57 +1,94 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace DiscordChatExporter.Core.Markdown.Parsing;
internal abstract class MarkdownVisitor
{
protected virtual ValueTask<MarkdownNode> VisitTextAsync(TextNode text) =>
protected virtual ValueTask<MarkdownNode> VisitTextAsync(
TextNode text,
CancellationToken cancellationToken = default) =>
new(text);
protected virtual async ValueTask<MarkdownNode> VisitFormattingAsync(FormattingNode formatting)
protected virtual async ValueTask<MarkdownNode> VisitFormattingAsync(
FormattingNode formatting,
CancellationToken cancellationToken = default)
{
await VisitAsync(formatting.Children);
await VisitAsync(formatting.Children, cancellationToken);
return formatting;
}
protected virtual ValueTask<MarkdownNode> VisitInlineCodeBlockAsync(InlineCodeBlockNode inlineCodeBlock) =>
protected virtual ValueTask<MarkdownNode> VisitInlineCodeBlockAsync(
InlineCodeBlockNode inlineCodeBlock,
CancellationToken cancellationToken = default) =>
new(inlineCodeBlock);
protected virtual ValueTask<MarkdownNode> VisitMultiLineCodeBlockAsync(MultiLineCodeBlockNode multiLineCodeBlock) =>
protected virtual ValueTask<MarkdownNode> VisitMultiLineCodeBlockAsync(
MultiLineCodeBlockNode multiLineCodeBlock,
CancellationToken cancellationToken = default) =>
new(multiLineCodeBlock);
protected virtual async ValueTask<MarkdownNode> VisitLinkAsync(LinkNode link)
protected virtual async ValueTask<MarkdownNode> VisitLinkAsync(
LinkNode link,
CancellationToken cancellationToken = default)
{
await VisitAsync(link.Children);
await VisitAsync(link.Children, cancellationToken);
return link;
}
protected virtual ValueTask<MarkdownNode> VisitEmojiAsync(EmojiNode emoji) =>
protected virtual ValueTask<MarkdownNode> VisitEmojiAsync(
EmojiNode emoji,
CancellationToken cancellationToken = default) =>
new(emoji);
protected virtual ValueTask<MarkdownNode> VisitMentionAsync(MentionNode mention) =>
protected virtual ValueTask<MarkdownNode> VisitMentionAsync(
MentionNode mention,
CancellationToken cancellationToken = default) =>
new(mention);
protected virtual ValueTask<MarkdownNode> VisitUnixTimestampAsync(UnixTimestampNode timestamp) =>
protected virtual ValueTask<MarkdownNode> VisitUnixTimestampAsync(
UnixTimestampNode timestamp,
CancellationToken cancellationToken = default) =>
new(timestamp);
public async ValueTask<MarkdownNode> VisitAsync(MarkdownNode node) => node switch
{
TextNode text => await VisitTextAsync(text),
FormattingNode formatting => await VisitFormattingAsync(formatting),
InlineCodeBlockNode inlineCodeBlock => await VisitInlineCodeBlockAsync(inlineCodeBlock),
MultiLineCodeBlockNode multiLineCodeBlock => await VisitMultiLineCodeBlockAsync(multiLineCodeBlock),
LinkNode link => await VisitLinkAsync(link),
EmojiNode emoji => await VisitEmojiAsync(emoji),
MentionNode mention => await VisitMentionAsync(mention),
UnixTimestampNode timestamp => await VisitUnixTimestampAsync(timestamp),
_ => throw new ArgumentOutOfRangeException(nameof(node))
};
public async ValueTask<MarkdownNode> VisitAsync(
MarkdownNode node,
CancellationToken cancellationToken = default) => node switch
{
TextNode text =>
await VisitTextAsync(text, cancellationToken),
public async ValueTask VisitAsync(IEnumerable<MarkdownNode> nodes)
FormattingNode formatting =>
await VisitFormattingAsync(formatting, cancellationToken),
InlineCodeBlockNode inlineCodeBlock =>
await VisitInlineCodeBlockAsync(inlineCodeBlock, cancellationToken),
MultiLineCodeBlockNode multiLineCodeBlock =>
await VisitMultiLineCodeBlockAsync(multiLineCodeBlock, cancellationToken),
LinkNode link =>
await VisitLinkAsync(link, cancellationToken),
EmojiNode emoji =>
await VisitEmojiAsync(emoji, cancellationToken),
MentionNode mention =>
await VisitMentionAsync(mention, cancellationToken),
UnixTimestampNode timestamp =>
await VisitUnixTimestampAsync(timestamp, cancellationToken),
_ => throw new ArgumentOutOfRangeException(nameof(node))
};
public async ValueTask VisitAsync(
IEnumerable<MarkdownNode> nodes,
CancellationToken cancellationToken = default)
{
foreach (var node in nodes)
await VisitAsync(node);
await VisitAsync(node, cancellationToken);
}
}
@@ -1,9 +1,5 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Net;
using System.Net.Http;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Utils.Extensions;
@@ -32,13 +28,4 @@ public static class ExceptionExtensions
PopulateChildren(exception, children);
return children;
}
public static HttpStatusCode? TryGetStatusCode(this HttpRequestException ex) =>
// This is extremely frail, but there's no other way
Regex
.Match(ex.Message, @": (\d+) \(")
.Groups[1]
.Value
.NullIfWhiteSpace()?
.Pipe(s => (HttpStatusCode) int.Parse(s, CultureInfo.InvariantCulture));
}
@@ -1,11 +1,24 @@
using System.Net.Http.Headers;
using System.Globalization;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Utils.Extensions;
public static class HttpExtensions
{
public static string? TryGetValue(this HttpContentHeaders headers, string name) =>
public static string? TryGetValue(this HttpHeaders headers, string name) =>
headers.TryGetValues(name, out var values)
? string.Concat(values)
: null;
public static HttpStatusCode? TryGetStatusCode(this HttpRequestException ex) =>
// This is extremely frail, but there's no other way
Regex
.Match(ex.Message, @": (\d+) \(")
.Groups[1]
.Value
.NullIfWhiteSpace()?
.Pipe(s => (HttpStatusCode) int.Parse(s, CultureInfo.InvariantCulture));
}
@@ -0,0 +1,17 @@
using System;
namespace DiscordChatExporter.Core.Utils.Extensions;
public static class TimeSpanExtensions
{
public static TimeSpan Clamp(this TimeSpan value, TimeSpan min, TimeSpan max)
{
if (value < min)
return min;
if (value > max)
return max;
return value;
}
}
+4 -11
View File
@@ -39,18 +39,11 @@ public static class Http
8,
(i, result, _) =>
{
// If rate-limited, use retry-after as a guide
if (result.Result?.StatusCode == HttpStatusCode.TooManyRequests)
// If rate-limited, use retry-after header as the guide
if (result.Result.Headers.RetryAfter?.Delta is { } retryAfter)
{
// Only start respecting retry-after after a few attempts, because
// Discord often sends unreasonable (20+ minutes) retry-after
// on the very first request.
if (i > 3)
{
var retryAfterDelay = result.Result.Headers.RetryAfter?.Delta;
if (retryAfterDelay is not null)
return retryAfterDelay.Value + TimeSpan.FromSeconds(1); // margin just in case
}
// Add some buffer just in case
return retryAfter + TimeSpan.FromSeconds(1);
}
return TimeSpan.FromSeconds(Math.Pow(2, i) + 1);
+9
View File
@@ -1,5 +1,6 @@
using System.Collections.Generic;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;
namespace DiscordChatExporter.Core.Utils;
@@ -15,6 +16,14 @@ public static class PathEx
foreach (var c in path)
buffer.Append(!InvalidFileNameChars.Contains(c) ? c : '_');
// File names cannot end with a dot on Windows
// https://github.com/Tyrrrz/DiscordChatExporter/issues/977
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
while (buffer.Length > 0 && buffer[^1] == '.')
buffer.Remove(buffer.Length - 1, 1);
}
return buffer.ToString();
}
+27 -23
View File
@@ -9,37 +9,41 @@ using System.Windows;
using System.Windows.Threading;
#endif
namespace DiscordChatExporter.Gui
namespace DiscordChatExporter.Gui;
public class Bootstrapper : Bootstrapper<RootViewModel>
{
public class Bootstrapper : Bootstrapper<RootViewModel>
protected override void OnStart()
{
protected override void OnStart()
{
base.OnStart();
base.OnStart();
// Set default theme
// (preferred theme will be chosen later, once the settings are loaded)
App.SetLightTheme();
}
// Set default theme
// (preferred theme will be set later, once the settings are loaded)
App.SetLightTheme();
}
protected override void ConfigureIoC(IStyletIoCBuilder builder)
{
base.ConfigureIoC(builder);
protected override void ConfigureIoC(IStyletIoCBuilder builder)
{
base.ConfigureIoC(builder);
// Bind settings as singleton
builder.Bind<SettingsService>().ToSelf().InSingletonScope();
// Bind settings as singleton
builder.Bind<SettingsService>().ToSelf().InSingletonScope();
// Bind view model factory
builder.Bind<IViewModelFactory>().ToAbstractFactory();
}
// Bind view model factory
builder.Bind<IViewModelFactory>().ToAbstractFactory();
}
#if !DEBUG
protected override void OnUnhandledException(DispatcherUnhandledExceptionEventArgs e)
{
base.OnUnhandledException(e);
protected override void OnUnhandledException(DispatcherUnhandledExceptionEventArgs e)
{
base.OnUnhandledException(e);
MessageBox.Show(e.Exception.ToString(), "Error occured", MessageBoxButton.OK, MessageBoxImage.Error);
}
#endif
MessageBox.Show(
e.Exception.ToString(),
"Error occured",
MessageBoxButton.OK,
MessageBoxImage.Error
);
}
#endif
}
@@ -1,8 +1,8 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>$(TargetFramework)-windows</TargetFramework>
<OutputType>WinExe</OutputType>
<TargetFramework>$(TargetFramework)-windows</TargetFramework>
<AssemblyName>DiscordChatExporter</AssemblyName>
<UseWPF>true</UseWPF>
<ApplicationIcon>../favicon.ico</ApplicationIcon>
@@ -18,11 +18,11 @@
<PackageReference Include="MaterialDesignThemes" Version="4.6.1" />
<PackageReference Include="Microsoft.Xaml.Behaviors.Wpf" Version="1.1.39" />
<PackageReference Include="Ookii.Dialogs.Wpf" Version="5.0.1" />
<PackageReference Include="Onova" Version="2.6.2" />
<PackageReference Include="Onova" Version="2.6.3" />
<PackageReference Include="Stylet" Version="1.3.6" />
<PackageReference Include="Tyrrrz.Settings" Version="1.3.4" />
<PackageReference Include="PropertyChanged.Fody" Version="4.0.5" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.3.1" PrivateAssets="all" />
<PackageReference Include="PropertyChanged.Fody" Version="4.1.0" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.4.0" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
+48
View File
@@ -0,0 +1,48 @@
using System;
using System.Globalization;
using System.Runtime.CompilerServices;
using System.Windows;
using Microsoft.Win32;
namespace DiscordChatExporter.Gui;
public static class Sanctions
{
[ModuleInitializer]
internal static void Verify()
{
var isSkipped = string.Equals(
Environment.GetEnvironmentVariable("RUSNI"),
"PYZDA",
StringComparison.OrdinalIgnoreCase
);
if (isSkipped)
return;
var locale = CultureInfo.CurrentCulture.Name;
var region = Registry.CurrentUser
.OpenSubKey(@"Control Panel\International\Geo", false)?
.GetValue("Name") as string;
var isSanctioned =
locale.EndsWith("-ru", StringComparison.OrdinalIgnoreCase) ||
locale.EndsWith("-by", StringComparison.OrdinalIgnoreCase) ||
string.Equals(region, "ru", StringComparison.OrdinalIgnoreCase) ||
string.Equals(region, "by", StringComparison.OrdinalIgnoreCase);
if (!isSanctioned)
return;
MessageBox.Show(
"You cannot use this software on the territory of a terrorist state. " +
"Set the environment variable `RUSNI=PYZDA` if you wish to override this check.",
"Sanctioned region",
MessageBoxButton.OK,
MessageBoxImage.Error
);
Environment.Exit(0xFACC);
}
}
@@ -17,9 +17,9 @@ public class RootViewModel : Screen, IHandle<NotificationMessage>, IDisposable
private readonly DialogManager _dialogManager;
private readonly SettingsService _settingsService;
private readonly UpdateService _updateService;
public SnackbarMessageQueue Notifications { get; } = new(TimeSpan.FromSeconds(5));
public DashboardViewModel Dashboard { get; }
public RootViewModel(
@@ -33,9 +33,9 @@ public class RootViewModel : Screen, IHandle<NotificationMessage>, IDisposable
_dialogManager = dialogManager;
_settingsService = settingsService;
_updateService = updateService;
eventAggregator.Subscribe(this);
Dashboard = _viewModelFactory.CreateDashboardViewModel();
DisplayName = $"{App.Name} v{App.VersionString}";
@@ -54,10 +54,10 @@ Press LEARN MORE to find ways that you can help.".Trim(),
if (await _dialogManager.ShowDialogAsync(dialog) == true)
{
ProcessEx.StartShellExecute("https://tyrrrz.me");
ProcessEx.StartShellExecute("https://tyrrrz.me/ukraine?source=discordchatexporter");
}
}
private async ValueTask CheckForUpdatesAsync()
{
try
@@ -84,7 +84,7 @@ Press LEARN MORE to find ways that you can help.".Trim(),
Notifications.Enqueue("Failed to perform application update");
}
}
public async void OnViewFullyLoaded()
{
await ShowWarInUkraineMessageAsync();
@@ -115,7 +115,7 @@ Press LEARN MORE to find ways that you can help.".Trim(),
_updateService.FinalizeUpdate(false);
}
public void Handle(NotificationMessage message) =>
public void Handle(NotificationMessage message) =>
Notifications.Enqueue(message.Text);
public void Dispose() => Notifications.Dispose();
@@ -181,7 +181,7 @@
<LineBreak />
<Run Text="7. Select the" />
<Run FontWeight="SemiBold" Text="Headers" />
<Run Text=" tab on the right" />
<Run Text="tab on the right" />
<LineBreak />
<Run Text="8. Scroll down to the" />
<Run FontWeight="SemiBold" Text="Request Headers" />
+1 -1
View File
@@ -1,5 +1,5 @@
DiscordChatExporter
Copyright (C) 2017-2022 Oleksii Holub
Copyright (C) 2017-2023 Oleksii Holub
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
+5 -5
View File
@@ -1,7 +1,7 @@
# DiscordChatExporter
[![Made in Ukraine](https://img.shields.io/badge/made_in-ukraine-ffd700.svg?labelColor=0057b7)](https://vshymanskyy.github.io/StandWithUkraine)
[![Build](https://img.shields.io/github/workflow/status/Tyrrrz/DiscordChatExporter/main/master)](https://github.com/Tyrrrz/DiscordChatExporter/actions)
[![Build](https://img.shields.io/github/actions/workflow/status/Tyrrrz/DiscordChatExporter/main.yml?branch=master)](https://github.com/Tyrrrz/DiscordChatExporter/actions)
[![Coverage](https://img.shields.io/codecov/c/github/Tyrrrz/DiscordChatExporter/master)](https://codecov.io/gh/Tyrrrz/DiscordChatExporter)
[![Release](https://img.shields.io/github/release/Tyrrrz/DiscordChatExporter.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
[![Downloads](https://img.shields.io/github/downloads/Tyrrrz/DiscordChatExporter/total.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
@@ -84,12 +84,12 @@ The following table lists all available download options:
> If you have any issues with them, please contact the corresponding maintainers.
> **Warning**:
> To run **DiscordChatExporter** on macOS and Linux, you need to make sure that **.NET Runtime v6** is installed.
> To run **DiscordChatExporter** on macOS and Linux, you need to make sure that **.NET 7.0 Runtime** is installed.
> You can download it here:
>
> - [.NET Runtime v6 for **macOS x64**](https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-6.0.6-macos-x64-installer)
> - [.NET Runtime v6 for **macOS Arm64**](https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-6.0.6-macos-arm64-installer)
> - [.NET Runtime v6 for **Linux**](https://docs.microsoft.com/en-us/dotnet/core/install/linux) (find the correct download for your distro)
> - [.NET 7.0 Runtime for **macOS x64**](https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/sdk-7.0.101-macos-x64-installer)
> - [.NET 7.0 Runtime for **macOS Arm64**](https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/sdk-7.0.101-macos-arm64-installer)
> - [.NET 7.0 Runtime for **Linux**](https://docs.microsoft.com/en-us/dotnet/core/install/linux) (find the correct download for your distro)
>
> This should not be necessary if you install **DiscordChatExporter** using a package manager, as it should take care of the dependencies for you.
> This is also not necessary if you are running **DiscordChatExporter** via Docker, because the image already contains the runtime.