Compare commits

..

24 Commits

Author SHA1 Message Date
Tyrrrz 966627f289 Update version 2023-03-10 16:35:13 +02:00
Tyrrrz 4ec00d0ec8 Clean up CSS 2023-03-10 16:33:56 +02:00
Tyrrrz 8bc1bcaf16 Render linked videos as embeds in HTML
Related to #1012
2023-03-10 16:17:35 +02:00
Tyrrrz 8f40acdda7 Load mentions for users that left the guild 2023-03-08 12:32:59 +02:00
Mark Cilia Vincenti bbe18368fa Update to AsyncKeyedLock 6.2.0 (#1003) 2023-02-25 20:30:00 +02:00
Tyrrrz 48eece554c Update version 2023-02-25 00:32:14 +02:00
Tyrrrz 7d7f1cc1e5 Update NuGet packages 2023-02-25 00:16:08 +02:00
Tyrrrz 2a1998b81c Add support for group DMs in invite embeds 2023-02-23 19:33:49 +02:00
Tyrrrz ac7f08ff3e Remove redundant explicit RazorBlade build references 2023-02-23 06:15:56 +02:00
Tyrrrz 95c5a43f6e Fix more sticker tests 2023-02-23 01:05:26 +02:00
Tyrrrz 91afded824 Fix sticker tests 2023-02-23 01:04:29 +02:00
Tyrrrz 0981c51fb9 Fix nick fallback 2023-02-23 01:03:25 +02:00
Tyrrrz 55209a0517 Add support for member-level avatars 2023-02-23 00:54:02 +02:00
Tyrrrz c2c35cf3a3 Treat gifv attachments as videos instead of images 2023-02-23 00:45:58 +02:00
Tyrrrz 404542d973 Replace Tyrrrz.Settings with Cogwheel 2023-02-20 02:56:39 +02:00
Tyrrrz f36852a1a2 Quote font URLs in HTML preamble to avoid issues 2023-02-18 20:01:46 +02:00
Tyrrrz 0e1c3e4c76 Add support for extracting channels from data dump in exportall
Closes #597
2023-02-18 19:45:31 +02:00
Tyrrrz 83e3289ead Use the new gg sans font 2023-02-18 18:22:27 +02:00
Tyrrrz b51900ec6a Update readme 2023-02-18 18:12:17 +02:00
Tyrrrz d1647e8286 Add support for selectable assets directory in GUI 2023-02-17 21:30:10 +02:00
Lucas LaBuff 95115f3e99 Add option to specify download directory for assets (#989)
Co-authored-by: Oleksii Holub <1935960+Tyrrrz@users.noreply.github.com>
2023-02-17 20:48:20 +02:00
Tyrrrz 53ff8ba430 Update screenshot 2023-02-16 23:16:04 +02:00
Tyrrrz cbab5fb71c Update changelog again 2023-02-16 23:08:48 +02:00
Tyrrrz 9512729ec2 Update changelog 2023-02-16 22:58:37 +02:00
47 changed files with 654 additions and 330 deletions
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

After

Width:  |  Height:  |  Size: 19 KiB

+21 -4
View File
@@ -1,18 +1,35 @@
# Changelog
## v2.39.1 (10-Mar-2023)
- Fixed an issue where messages containing mentions of users that left the server (but haven't deleted their account) were incorrectly rendered as `@Unknown`.
- Fixed an issue where messages containing video links were not rendered correctly in the HTML export. Now, they are rendered using a video player and the referenced assets are pulled if the "download assets" option is enabled.
## v2.39 (25-Feb-2023)
- Added an option to specify the directory path for downloaded assets. This can be used, in combination with the "reuse assets" option, to have a shared asset directory for multiple exports. In GUI, this option can be found in the advanced setion of the export dialog. In CLI, it's available as `--media-dir <path>`. (Thanks [@96-LB](https://github.com/96-LB))
- Added support for server-level user avatars. If a user has a custom avatar set on the server, it will be rendered instead of their main avatar.
- CLI changes:
- Extended the `exportall` command with the `--data-package` option. You can use this option to provide the path to your personal data package (`package.zip` file), requested from Discord. When provided, the command will pull channels listed in the package instead of using the Discord API to query them. This can be particularly useful if you want to export DM channels that have been closed, as the personal data package should still contain their IDs.
- HTML format changes:
- Added support for rendering group DM channel invites.
- Changed the primary font from `Whitney` to `gg sans` to match the new Discord design.
- Fixed an issue where the external font sometimes failed to load if the export was created with the "download assets" option enabled.
- Fixed an issue where `gifv` attachments were incorrectly treated as images instead of video files.
## v2.38 (16-Feb-2023)
- General changes:
- Added an option to disable markdown processing when exporting. You can use this to produce JSON or plain text exports without unwrapping mentions, custom emoji, and certain other special tokens. In GUI, this option can be found in the export dialog. In CLI, it's available as the `--markdown <true|false>` flag.
- Added support for different formatting options when rendering timestamps (i.e. the `f` in `<t:1234567890:f>`). As an exception, relative timestamps are formatted as regular timestamps since that makes more sense in a static export. Currently, all formatting options are rendered based on the `English (US)` locale, with the intent to make that more configurable in the future.
- Added an option to disable markdown processing when exporting. You can use this to produce JSON or plain text exports without unwrapping mentions, custom emoji, and certain other special tokens.
- Moved the "reuse assets" option from the settings dialog to the export dialog.
- Changed the default date format from `dd-MMM-yy hh:mm tt` to `MM/dd/yyyy h:mm tt` to match the date format used by Discord for message timestamps when the `English (US)` language is selected. This should make the default date format more consistent with other non-configurable date formats used by DiscordChatExporter. In the future, this setting will most likely be removed entirely in favor of a more flexible approach.
- Updated the usage guide with additional steps required to retrieve the user token.
- Fixed an issue where certain user mentions were incorrectly rendered as `@Unknown`.
- Fixed an issue where some embed images were rendered using their actual URLs instead of Discord proxy URLs.
- GUI changes:
- Added a text box for specifying the output path when exporting channels. It can be used to set an output path that uses template tokens, which previously required hacky workarounds. This text box can be left empty, in which case clicking on the "Export" button will open the file or folder picker dialog, matching the current behavior.
- Added a text box for specifying the output path when exporting channels. It can be used to set an output path that uses template tokens, which previously required hacky workarounds. This text box can be left empty, in which case clicking on the "Export" button will open the file or folder picker dialog, matching the previous behavior.
- Added a notification when the application is updated to a new version, containing the changelog link.
- Moved the "reuse assets" option from the settings dialog to the export dialog.
- Replaced the lock icon in the token text box with a key icon, to better reflect the purpose of the input.
- Minor cosmetic improvements.
- CLI changes:
@@ -20,7 +37,7 @@
- HTML format changes:
- Added proper support for more system notification messages.
- Added support for rendering server invites. Now, if the message contains an invite link, the export will show the server icon and include additional information, such as the name of the server and the target channel.
- Changed the preamble part of the export to show the channel icon when available, instead of the default Discord logo. This is only available for group DM channels.
- Changed the preamble part of the export to show the channel icon when available, instead of the default Discord logo. This is only available for group DM channels. (Thanks [@CanePlayz](https://github.com/CanePlayz))
- Fixed an issue where some miscellaneous characters were incorrectly treated as emoji, which resulted in them being rendered as broken images. (Thanks [@CanePlayz](https://github.com/CanePlayz))
- Fixed an issue where messages triggered by app interactions were not rendered as replies.
- Fixed an issue where server cross-posts were not rendered correctly.
+1 -1
View File
@@ -2,7 +2,7 @@
<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<Version>2.38</Version>
<Version>2.39.1</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Oleksii Holub</Copyright>
<LangVersion>preview</LangVersion>
@@ -19,7 +19,7 @@
<PackageReference Include="Microsoft.Extensions.Configuration" Version="7.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="7.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="7.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.4.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.5.0" />
<PackageReference Include="ReflectionMagic" Version="4.1.0" />
<PackageReference Include="xunit" Version="2.4.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" PrivateAssets="all" />
@@ -3,6 +3,7 @@ using System.Threading.Tasks;
using AngleSharp.Dom;
using DiscordChatExporter.Cli.Tests.Infra;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Utils.Extensions;
using FluentAssertions;
using Xunit;
@@ -46,7 +47,8 @@ public class HtmlEmbedSpecs
message
.QuerySelectorAll("img")
.Select(e => e.GetAttribute("src"))
.Where(s => s?.EndsWith("i.redd.it/f8w05ja8s4e61.png") ?? false)
.WhereNotNull()
.Where(s => s.EndsWith("f8w05ja8s4e61.png"))
.Should()
.ContainSingle();
}
@@ -67,6 +69,25 @@ public class HtmlEmbedSpecs
content.Should().BeNullOrEmpty();
}
[Fact]
public async Task Message_with_a_video_link_is_rendered_with_a_video_embed()
{
// Act
var message = await ExportWrapper.GetMessageAsHtmlAsync(
ChannelIds.EmbedTestCases,
Snowflake.Parse("1083751036596002856")
);
// Assert
message
.QuerySelectorAll("source")
.Select(e => e.GetAttribute("src"))
.WhereNotNull()
.Where(s => s.EndsWith("i_am_currently_feeling_slight_displeasure_of_what_you_have_just_sent_lqrem.mp4"))
.Should()
.ContainSingle();
}
[Fact]
public async Task Message_with_a_GIFV_link_is_rendered_with_a_video_embed()
{
@@ -80,7 +101,8 @@ public class HtmlEmbedSpecs
message
.QuerySelectorAll("source")
.Select(e => e.GetAttribute("src"))
.Where(s => s?.EndsWith("media.tenor.com/DDAJeW6BQKkAAAPo/tooncasm-test-copy.mp4") ?? false)
.WhereNotNull()
.Where(s => s.EndsWith("tooncasm-test-copy.mp4"))
.Should()
.ContainSingle();
}
@@ -19,7 +19,7 @@ public class HtmlStickerSpecs
// Assert
var stickerUrl = message.QuerySelector("[title='rock'] img")?.GetAttribute("src");
stickerUrl.Should().Be("https://discord.com/stickers/904215665597120572.png");
stickerUrl.Should().Be("https://cdn.discordapp.com/stickers/904215665597120572.png");
}
[Fact]
@@ -33,6 +33,6 @@ public class HtmlStickerSpecs
// Assert
var stickerUrl = message.QuerySelector("[title='Yikes'] [data-source]")?.GetAttribute("data-source");
stickerUrl.Should().Be("https://discord.com/stickers/816087132447178774.json");
stickerUrl.Should().Be("https://cdn.discordapp.com/stickers/816087132447178774.json");
}
}
@@ -26,8 +26,8 @@ public class JsonStickerSpecs
sticker.GetProperty("id").GetString().Should().Be("904215665597120572");
sticker.GetProperty("name").GetString().Should().Be("rock");
sticker.GetProperty("format").GetString().Should().Be("PngAnimated");
sticker.GetProperty("sourceUrl").GetString().Should().Be("https://discord.com/stickers/904215665597120572.png");
sticker.GetProperty("format").GetString().Should().Be("Apng");
sticker.GetProperty("sourceUrl").GetString().Should().Be("https://cdn.discordapp.com/stickers/904215665597120572.png");
}
[Fact]
@@ -48,6 +48,6 @@ public class JsonStickerSpecs
sticker.GetProperty("id").GetString().Should().Be("816087132447178774");
sticker.GetProperty("name").GetString().Should().Be("Yikes");
sticker.GetProperty("format").GetString().Should().Be("Lottie");
sticker.GetProperty("sourceUrl").GetString().Should().Be("https://discord.com/stickers/816087132447178774.json");
sticker.GetProperty("sourceUrl").GetString().Should().Be("https://cdn.discordapp.com/stickers/816087132447178774.json");
}
}
@@ -7,7 +7,7 @@ using DiscordChatExporter.Core.Discord;
namespace DiscordChatExporter.Cli.Commands.Base;
public abstract class TokenCommandBase : ICommand
public abstract class DiscordCommandBase : ICommand
{
[CommandOption(
"token",
@@ -20,7 +20,7 @@ using Gress;
namespace DiscordChatExporter.Cli.Commands.Base;
public abstract class ExportCommandBase : TokenCommandBase
public abstract class ExportCommandBase : DiscordCommandBase
{
private readonly string _outputPath = Directory.GetCurrentDirectory();
@@ -98,6 +98,20 @@ public abstract class ExportCommandBase : TokenCommandBase
)]
public bool ShouldReuseAssets { get; init; }
private readonly string? _assetsDirPath;
[CommandOption(
"media-dir",
Description = "Download assets to this directory. If not specified, the asset directory path will be derived from the output path."
)]
public string? AssetsDirPath
{
get => _assetsDirPath;
// Handle ~/ in paths on Unix systems
// https://github.com/Tyrrrz/DiscordChatExporter/pull/903
init => _assetsDirPath = value is not null ? Path.GetFullPath(value) : null;
}
[CommandOption(
"dateformat",
Description = "Format used when writing dates."
@@ -124,6 +138,14 @@ public abstract class ExportCommandBase : TokenCommandBase
);
}
// Assets directory should only be specified when the download assets option is set
if (!string.IsNullOrWhiteSpace(AssetsDirPath) && !ShouldDownloadAssets)
{
throw new CommandException(
"Option --media-dir cannot be used without --media."
);
}
// Make sure the user does not try to export all channels into a single file.
// Output path must either be a directory, or contain template tokens.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/799
@@ -172,6 +194,7 @@ public abstract class ExportCommandBase : TokenCommandBase
guild,
channel,
OutputPath,
AssetsDirPath,
ExportFormat,
After,
Before,
@@ -245,7 +268,7 @@ public abstract class ExportCommandBase : TokenCommandBase
await console.Output.WriteLineAsync("Resolving channel(s)...");
var channels = new List<Channel>();
var guildChannelMap = new Dictionary<Snowflake, IReadOnlyList<Channel>>();
var channelsByGuild = new Dictionary<Snowflake, IReadOnlyList<Channel>>();
foreach (var channelId in channelIds)
{
@@ -255,7 +278,7 @@ public abstract class ExportCommandBase : TokenCommandBase
if (channel.Kind == ChannelKind.GuildCategory)
{
var guildChannels =
guildChannelMap.GetValueOrDefault(channel.GuildId) ??
channelsByGuild.GetValueOrDefault(channel.GuildId) ??
await Discord.GetGuildChannelsAsync(channel.GuildId, cancellationToken);
foreach (var guildChannel in guildChannels)
@@ -265,7 +288,7 @@ public abstract class ExportCommandBase : TokenCommandBase
}
// Cache the guild channels to avoid redundant work
guildChannelMap[channel.GuildId] = guildChannels;
channelsByGuild[channel.GuildId] = guildChannels;
}
else
{
@@ -1,9 +1,15 @@
using System.Collections.Generic;
using System.IO.Compression;
using System.Text.Json;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Exceptions;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exceptions;
using JsonExtensions.Reading;
namespace DiscordChatExporter.Cli.Commands;
@@ -14,7 +20,19 @@ public class ExportAllCommand : ExportCommandBase
"include-dm",
Description = "Include direct message channels."
)]
public bool IncludeDirectMessages { get; init; } = true;
public bool IncludeDirectChannels { get; init; } = true;
[CommandOption(
"include-guilds",
Description = "Include guild channels."
)]
public bool IncludeGuildChannels { get; init; } = true;
[CommandOption(
"data-package",
Description = "Path to the personal data package (ZIP file) requested from Discord. If provided, only channels referenced in the dump will be exported."
)]
public string? DataPackageFilePath { get; init; }
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -23,16 +41,60 @@ public class ExportAllCommand : ExportCommandBase
var cancellationToken = console.RegisterCancellationHandler();
var channels = new List<Channel>();
await console.Output.WriteLineAsync("Fetching channels...");
await foreach (var guild in Discord.GetUserGuildsAsync(cancellationToken))
// Pull from the API
if (string.IsNullOrWhiteSpace(DataPackageFilePath))
{
// Skip DMs if instructed to
if (!IncludeDirectMessages && guild.Id == Guild.DirectMessages.Id)
continue;
await console.Output.WriteLineAsync("Fetching channels...");
await foreach (var channel in Discord.GetGuildChannelsAsync(guild.Id, cancellationToken))
channels.Add(channel);
await foreach (var guild in Discord.GetUserGuildsAsync(cancellationToken))
{
await foreach (var channel in Discord.GetGuildChannelsAsync(guild.Id, cancellationToken))
{
channels.Add(channel);
}
}
}
// Pull from the data package
else
{
await console.Output.WriteLineAsync("Extracting channels...");
using var archive = ZipFile.OpenRead(DataPackageFilePath);
var entry = archive.GetEntry("messages/index.json");
if (entry is null)
throw new CommandException("Cannot find channel index inside the data package.");
await using var stream = entry.Open();
using var document = await JsonDocument.ParseAsync(stream, default, cancellationToken);
foreach (var property in document.RootElement.EnumerateObjectOrEmpty())
{
var channelId = Snowflake.Parse(property.Name);
var channelName = property.Value.GetString();
// Null items refer to deleted channels
if (channelName is null)
continue;
await console.Output.WriteLineAsync($"Fetching channel '{channelName}' ({channelId})...");
try
{
var channel = await Discord.GetChannelAsync(channelId, cancellationToken);
channels.Add(channel);
}
catch (DiscordChatExporterException)
{
await console.Output.WriteLineAsync($"Channel '{channelName}' ({channelId}) is inaccessible.");
}
}
}
// Filter out unwanted channels
if (!IncludeDirectChannels)
channels.RemoveAll(c => c.Kind.IsDirect());
if (!IncludeGuildChannels)
channels.RemoveAll(c => c.Kind.IsGuild());
await base.ExecuteAsync(console, channels);
}
@@ -14,7 +14,7 @@ public class ExportChannelsCommand : ExportCommandBase
[CommandOption(
"channel",
'c',
Description = "Channel ID(s). If provided with a category ID, all channels in that category will be exported."
Description = "Channel ID(s). If provided with category IDs, all channels inside those categories will be exported."
)]
public required IReadOnlyList<Snowflake> ChannelIds { get; init; }
@@ -1,5 +1,4 @@
using System.Linq;
using System.Threading.Tasks;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
@@ -18,10 +17,7 @@ public class ExportDirectMessagesCommand : ExportCommandBase
var cancellationToken = console.RegisterCancellationHandler();
await console.Output.WriteLineAsync("Fetching channels...");
var channels = (await Discord.GetGuildChannelsAsync(Guild.DirectMessages.Id, cancellationToken))
.Where(c => c.Kind != ChannelKind.GuildCategory)
.ToArray();
var channels = await Discord.GetGuildChannelsAsync(Guild.DirectMessages.Id, cancellationToken);
await base.ExecuteAsync(console, channels);
}
@@ -11,7 +11,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("channels", Description = "Get the list of channels in a guild.")]
public class GetChannelsCommand : TokenCommandBase
public class GetChannelsCommand : DiscordCommandBase
{
[CommandOption(
"guild",
@@ -10,7 +10,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("dm", Description = "Get the list of direct message channels.")]
public class GetDirectMessageChannelsCommand : TokenCommandBase
public class GetDirectChannelsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -10,7 +10,7 @@ using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Cli.Commands;
[Command("guilds", Description = "Get the list of accessible guilds.")]
public class GetGuildsCommand : TokenCommandBase
public class GetGuildsCommand : DiscordCommandBase
{
public override async ValueTask ExecuteAsync(IConsole console)
{
@@ -2,7 +2,6 @@
using System.IO;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -20,11 +19,26 @@ public partial record Attachment(
{
public string FileExtension => Path.GetExtension(FileName);
public bool IsImage => FileFormat.IsImage(FileExtension);
public bool IsImage =>
string.Equals(FileExtension, ".jpg", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".jpeg", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".png", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".gif", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".bmp", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".webp", StringComparison.OrdinalIgnoreCase);
public bool IsVideo => FileFormat.IsVideo(FileExtension);
public bool IsVideo =>
string.Equals(FileExtension, ".gifv", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".mp4", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".webm", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".mov", StringComparison.OrdinalIgnoreCase);
public bool IsAudio => FileFormat.IsAudio(FileExtension);
public bool IsAudio =>
string.Equals(FileExtension, ".mp3", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".wav", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".ogg", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".flac", StringComparison.OrdinalIgnoreCase) ||
string.Equals(FileExtension, ".m4a", StringComparison.OrdinalIgnoreCase);
public bool IsSpoiler => FileName.StartsWith("SPOILER_", StringComparison.Ordinal);
}
@@ -1,5 +1,4 @@
using System;
using System.Linq;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
@@ -24,33 +23,16 @@ public partial record Channel(
public partial record Channel
{
private static ChannelCategory GetFallbackCategory(ChannelKind channelKind) => new(
Snowflake.Zero,
channelKind switch
{
ChannelKind.GuildTextChat => "Text",
ChannelKind.DirectTextChat => "Private",
ChannelKind.DirectGroupTextChat => "Group",
ChannelKind.GuildNews => "News",
_ => "Default"
},
null
);
private static string GetIconUrl(Snowflake id, string iconHash)
{
var extension = iconHash.StartsWith("a_", StringComparison.Ordinal)
? "gif"
: "png";
return $"https://cdn.discordapp.com/channel-icons/{id}/{iconHash}.{extension}";
}
public static Channel Parse(JsonElement json, ChannelCategory? category = null, int? positionHint = null)
public static Channel Parse(JsonElement json, ChannelCategory? categoryHint = null, int? positionHint = null)
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var kind = (ChannelKind)json.GetProperty("type").GetInt32();
var guildId = json.GetPropertyOrNull("guild_id")?.GetNonWhiteSpaceStringOrNull()?.Pipe(Snowflake.Parse);
var guildId =
json.GetPropertyOrNull("guild_id")?.GetNonWhiteSpaceStringOrNull()?.Pipe(Snowflake.Parse) ??
Guild.DirectMessages.Id;
var category = categoryHint ?? ChannelCategory.CreateDefault(kind);
var name =
// Guild channel
@@ -70,8 +52,11 @@ public partial record Channel
positionHint ??
json.GetPropertyOrNull("position")?.GetInt32OrNull();
// Only available on group DMs
var iconUrl = json.GetPropertyOrNull("icon")?.GetNonWhiteSpaceStringOrNull()?.Pipe(h => GetIconUrl(id, h));
// Icons can only be set for group DM channels
var iconUrl = json
.GetPropertyOrNull("icon")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(h => ImageCdn.GetChannelIconUrl(id, h));
var topic = json.GetPropertyOrNull("topic")?.GetStringOrNull();
@@ -83,8 +68,8 @@ public partial record Channel
return new Channel(
id,
kind,
guildId ?? Guild.DirectMessages.Id,
category ?? GetFallbackCategory(kind),
guildId,
category,
name,
position,
iconUrl,
@@ -7,7 +7,18 @@ namespace DiscordChatExporter.Core.Discord.Data;
public record ChannelCategory(Snowflake Id, string Name, int? Position) : IHasId
{
public static ChannelCategory Unknown { get; } = new(Snowflake.Zero, "<unknown category>", 0);
public static ChannelCategory CreateDefault(ChannelKind channelKind) => new(
Snowflake.Zero,
channelKind switch
{
ChannelKind.GuildTextChat => "Text",
ChannelKind.DirectTextChat => "Private",
ChannelKind.DirectGroupTextChat => "Group",
ChannelKind.GuildNews => "News",
_ => "Default"
},
null
);
public static ChannelCategory Parse(JsonElement json, int? positionHint = null)
{
@@ -16,4 +16,13 @@ public enum ChannelKind
GuildStageVoice = 13,
GuildDirectory = 14,
GuildForum = 15
}
public static class ChannelKindExtensions
{
public static bool IsDirect(this ChannelKind kind) =>
kind is ChannelKind.DirectTextChat or ChannelKind.DirectGroupTextChat;
public static bool IsGuild(this ChannelKind kind) =>
!kind.IsDirect();
}
@@ -0,0 +1,60 @@
using System;
using System.Linq;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Core.Discord.Data.Common;
// https://discord.com/developers/docs/reference#image-formatting
public static class ImageCdn
{
// Standard emoji are rendered through Twemoji
public static string GetStandardEmojiUrl(string emojiName)
{
var runes = emojiName.GetRunes().ToArray();
// Variant selector rune is skipped in Twemoji IDs,
// except when the emoji also contains a zero-width joiner.
// VS = 0xfe0f; ZWJ = 0x200d.
var filteredRunes = runes.Any(r => r.Value == 0x200d)
? runes
: runes.Where(r => r.Value != 0xfe0f);
var twemojiId = string.Join(
"-",
filteredRunes.Select(r => r.Value.ToString("x"))
);
return $"https://cdn.jsdelivr.net/gh/twitter/twemoji@latest/assets/svg/{twemojiId}.svg";
}
public static string GetCustomEmojiUrl(Snowflake emojiId, bool isAnimated = false) =>
isAnimated
? $"https://cdn.discordapp.com/emojis/{emojiId}.gif"
: $"https://cdn.discordapp.com/emojis/{emojiId}.png";
public static string GetGuildIconUrl(Snowflake guildId, string iconHash, int size = 512) =>
iconHash.StartsWith("a_", StringComparison.Ordinal)
? $"https://cdn.discordapp.com/icons/{guildId}/{iconHash}.gif?size={size}"
: $"https://cdn.discordapp.com/icons/{guildId}/{iconHash}.png?size={size}";
public static string GetChannelIconUrl(Snowflake channelId, string iconHash, int size = 512) =>
iconHash.StartsWith("a_", StringComparison.Ordinal)
? $"https://cdn.discordapp.com/channel-icons/{channelId}/{iconHash}.gif?size={size}"
: $"https://cdn.discordapp.com/channel-icons/{channelId}/{iconHash}.png?size={size}";
public static string GetUserAvatarUrl(Snowflake userId, string avatarHash, int size = 512) =>
avatarHash.StartsWith("a_", StringComparison.Ordinal)
? $"https://cdn.discordapp.com/avatars/{userId}/{avatarHash}.gif?size={size}"
: $"https://cdn.discordapp.com/avatars/{userId}/{avatarHash}.png?size={size}";
public static string GetFallbackUserAvatarUrl(int discriminator) =>
$"https://cdn.discordapp.com/embed/avatars/{discriminator % 5}.png";
public static string GetMemberAvatarUrl(Snowflake guildId, Snowflake userId, string avatarHash, int size = 512) =>
avatarHash.StartsWith("a_", StringComparison.Ordinal)
? $"https://cdn.discordapp.com/guilds/{guildId}/users/{userId}/avatars/{avatarHash}.gif?size={size}"
: $"https://cdn.discordapp.com/guilds/{guildId}/users/{userId}/avatars/{avatarHash}.png?size={size}";
public static string GetStickerUrl(Snowflake stickerId, string format = "png") =>
$"https://cdn.discordapp.com/stickers/{stickerId}.{format}";
}
+8 -29
View File
@@ -1,6 +1,6 @@
using System;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -24,38 +24,15 @@ public partial record Emoji(
public partial record Emoji
{
private static string GetTwemojiId(string name)
{
var runes = name.GetRunes().ToArray();
// Variant selector rune is skipped in Twemoji names, except when the emoji also contains a zero-width joiner.
// VS = 0xfe0f; ZWJ = 0x200d.
var filteredRunes = runes.Any(r => r.Value == 0x200d)
? runes
: runes.Where(r => r.Value != 0xfe0f);
return string.Join(
"-",
filteredRunes.Select(r => r.Value.ToString("x"))
);
}
private static string GetImageUrl(Snowflake id, bool isAnimated) => isAnimated
? $"https://cdn.discordapp.com/emojis/{id}.gif"
: $"https://cdn.discordapp.com/emojis/{id}.png";
private static string GetImageUrl(string name) =>
$"https://cdn.jsdelivr.net/gh/twitter/twemoji@latest/assets/svg/{GetTwemojiId(name)}.svg";
public static string GetImageUrl(Snowflake? id, string? name, bool isAnimated)
{
// Custom emoji
if (id is not null)
return GetImageUrl(id.Value, isAnimated);
return ImageCdn.GetCustomEmojiUrl(id.Value, isAnimated);
// Standard emoji
if (!string.IsNullOrWhiteSpace(name))
return GetImageUrl(name);
return ImageCdn.GetStandardEmojiUrl(name);
// Either ID or name should be set
throw new ApplicationException("Emoji has neither ID nor name set.");
@@ -64,14 +41,16 @@ public partial record Emoji
public static Emoji Parse(JsonElement json)
{
var id = json.GetPropertyOrNull("id")?.GetNonWhiteSpaceStringOrNull()?.Pipe(Snowflake.Parse);
var name = json.GetPropertyOrNull("name")?.GetNonWhiteSpaceStringOrNull();
// Names may be missing on custom emoji within reactions
var name = json.GetPropertyOrNull("name")?.GetNonWhiteSpaceStringOrNull() ?? "Unknown Emoji";
var isAnimated = json.GetPropertyOrNull("animated")?.GetBooleanOrNull() ?? false;
var imageUrl = GetImageUrl(id, name, isAnimated);
return new Emoji(
id,
// Name may be missing if it's an emoji inside a reaction
name ?? "<unknown emoji>",
name,
isAnimated,
imageUrl
);
+8 -17
View File
@@ -1,5 +1,4 @@
using System;
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -9,32 +8,24 @@ namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/guild#guild-object
public record Guild(Snowflake Id, string Name, string IconUrl) : IHasId
{
// Direct messages are encapsulated within a special pseudo-guild for consistency
public static Guild DirectMessages { get; } = new(
Snowflake.Zero,
"Direct Messages",
GetDefaultIconUrl()
ImageCdn.GetFallbackUserAvatarUrl(0)
);
private static string GetDefaultIconUrl() =>
"https://cdn.discordapp.com/embed/avatars/0.png";
private static string GetIconUrl(Snowflake id, string iconHash)
{
var extension = iconHash.StartsWith("a_", StringComparison.Ordinal)
? "gif"
: "png";
return $"https://cdn.discordapp.com/icons/{id}/{iconHash}.{extension}";
}
public static Guild Parse(JsonElement json)
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var name = json.GetProperty("name").GetNonNullString();
var iconUrl =
json.GetPropertyOrNull("icon")?.GetNonWhiteSpaceStringOrNull()?.Pipe(h => GetIconUrl(id, h)) ??
GetDefaultIconUrl();
json
.GetPropertyOrNull("icon")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(h => ImageCdn.GetGuildIconUrl(id, h)) ??
ImageCdn.GetFallbackUserAvatarUrl(0);
return new Guild(id, name, iconUrl);
}
@@ -8,7 +8,7 @@ namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/invite#invite-object
public record Invite(
string Code,
Guild? Guild,
Guild Guild,
Channel? Channel)
{
public static string? TryGetCodeFromUrl(string url) =>
@@ -17,7 +17,7 @@ public record Invite(
public static Invite Parse(JsonElement json)
{
var code = json.GetProperty("code").GetNonWhiteSpaceString();
var guild = json.GetPropertyOrNull("guild")?.Pipe(Guild.Parse);
var guild = json.GetPropertyOrNull("guild")?.Pipe(Guild.Parse) ?? Guild.DirectMessages;
var channel = json.GetPropertyOrNull("channel")?.Pipe(c => Channel.Parse(c));
return new Invite(code, guild, channel);
@@ -11,7 +11,8 @@ namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/guild#guild-member-object
public partial record Member(
User User,
string Nick,
string? Nick,
string? AvatarUrl,
IReadOnlyList<Snowflake> RoleIds) : IHasId
{
public Snowflake Id => User.Id;
@@ -19,7 +20,9 @@ public partial record Member(
public partial record Member
{
public static Member Parse(JsonElement json)
public static Member CreateDefault(User user) => new(user, null, null, Array.Empty<Snowflake>());
public static Member Parse(JsonElement json, Snowflake? guildId = null)
{
var user = json.GetProperty("user").Pipe(User.Parse);
var nick = json.GetPropertyOrNull("nick")?.GetNonWhiteSpaceStringOrNull();
@@ -31,9 +34,17 @@ public partial record Member
.Select(Snowflake.Parse)
.ToArray() ?? Array.Empty<Snowflake>();
var avatarUrl = guildId is not null
? json
.GetPropertyOrNull("avatar")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(h => ImageCdn.GetMemberAvatarUrl(guildId.Value, user.Id, h))
: null;
return new Member(
user,
nick ?? user.Name,
nick,
avatarUrl,
roleIds
);
}
@@ -1,4 +1,6 @@
using System.Text.Json;
using System;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -7,18 +9,19 @@ namespace DiscordChatExporter.Core.Discord.Data;
// https://discord.com/developers/docs/resources/sticker#sticker-resource
public record Sticker(Snowflake Id, string Name, StickerFormat Format, string SourceUrl)
{
private static string GetSourceUrl(Snowflake id, StickerFormat format)
{
var extension = format == StickerFormat.Lottie ? "json" : "png";
return $"https://discord.com/stickers/{id}.{extension}";
}
public static Sticker Parse(JsonElement json)
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
var name = json.GetProperty("name").GetNonNullString();
var format = (StickerFormat)json.GetProperty("format_type").GetInt32();
var sourceUrl = GetSourceUrl(id, format);
var sourceUrl = ImageCdn.GetStickerUrl(id, format switch
{
StickerFormat.Png => "png",
StickerFormat.Apng => "png",
StickerFormat.Lottie => "json",
_ => throw new InvalidOperationException($"Unknown sticker format '{format}'.")
});
return new Sticker(id, name, format, sourceUrl);
}
@@ -3,6 +3,6 @@
public enum StickerFormat
{
Png = 1,
PngAnimated = 2,
Apng = 2,
Lottie = 3
}
+6 -16
View File
@@ -1,5 +1,4 @@
using System;
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
@@ -21,18 +20,6 @@ public partial record User(
public partial record User
{
private static string GetDefaultAvatarUrl(int discriminator) =>
$"https://cdn.discordapp.com/embed/avatars/{discriminator % 5}.png";
private static string GetAvatarUrl(Snowflake id, string avatarHash)
{
var extension = avatarHash.StartsWith("a_", StringComparison.Ordinal)
? "gif"
: "png";
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.{extension}?size=512";
}
public static User Parse(JsonElement json)
{
var id = json.GetProperty("id").GetNonWhiteSpaceString().Pipe(Snowflake.Parse);
@@ -41,8 +28,11 @@ public partial record User
var name = json.GetProperty("username").GetNonNullString();
var avatarUrl =
json.GetPropertyOrNull("avatar")?.GetNonWhiteSpaceStringOrNull()?.Pipe(h => GetAvatarUrl(id, h)) ??
GetDefaultAvatarUrl(discriminator);
json
.GetPropertyOrNull("avatar")?
.GetNonWhiteSpaceStringOrNull()?
.Pipe(h => ImageCdn.GetUserAvatarUrl(id, h)) ??
ImageCdn.GetFallbackUserAvatarUrl(discriminator);
return new User(id, isBot, discriminator, name, avatarUrl);
}
@@ -150,6 +150,14 @@ public class DiscordClient
: null;
}
public async ValueTask<User?> TryGetUserAsync(
Snowflake userId,
CancellationToken cancellationToken = default)
{
var response = await TryGetJsonResponseAsync($"users/{userId}", cancellationToken);
return response?.Pipe(User.Parse);
}
public async IAsyncEnumerable<Guild> GetUserGuildsAsync(
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
@@ -260,7 +268,7 @@ public class DiscordClient
return null;
var response = await TryGetJsonResponseAsync($"guilds/{guildId}/members/{memberId}", cancellationToken);
return response?.Pipe(Member.Parse);
return response?.Pipe(j => Member.Parse(j, guildId));
}
public async ValueTask<Invite?> TryGetGuildInviteAsync(
@@ -284,7 +292,7 @@ public class DiscordClient
// Instead, we use an empty channel category as a fallback.
catch (DiscordChatExporterException)
{
return ChannelCategory.Unknown;
return new ChannelCategory(channelId, "Unknown Category", 0);
}
}
@@ -5,7 +5,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="AsyncKeyedLock" Version="6.1.1" />
<PackageReference Include="AsyncKeyedLock" Version="6.2.0" />
<PackageReference Include="Gress" Version="2.0.1" />
<PackageReference Include="JsonExtensions" Version="1.2.0" />
<PackageReference Include="Polly" Version="7.2.3" />
@@ -14,8 +14,4 @@
<PackageReference Include="WebMarkupMin.Core" Version="2.13.8" />
</ItemGroup>
<ItemGroup>
<RazorBlade Include="**\*.cshtml" />
</ItemGroup>
</Project>
</Project>
@@ -34,7 +34,7 @@ public class ChannelExporter
{
// Resolve members for referenced users
foreach (var user in message.GetReferencedUsers())
await context.PopulateMemberAsync(user.Id, cancellationToken);
await context.PopulateMemberAsync(user, cancellationToken);
// Export the message
if (request.MessageFilter.IsMatch(message))
@@ -20,6 +20,7 @@ internal class ExportContext
private readonly ExportAssetDownloader _assetDownloader;
public DiscordClient Discord { get; }
public ExportRequest Request { get; }
public ExportContext(DiscordClient discord,
@@ -29,7 +30,7 @@ internal class ExportContext
Request = request;
_assetDownloader = new ExportAssetDownloader(
request.OutputAssetsDirPath,
request.AssetsDirPath,
request.ShouldReuseAssets
);
}
@@ -43,18 +44,35 @@ internal class ExportContext
_roles[role.Id] = role;
}
// Because members are not pulled in bulk, we need to populate them on demand
public async ValueTask PopulateMemberAsync(Snowflake id, CancellationToken cancellationToken = default)
// Because members cannot be pulled in bulk, we need to populate them on demand
private async ValueTask PopulateMemberAsync(
Snowflake id,
User? fallbackUser,
CancellationToken cancellationToken = default)
{
if (_members.ContainsKey(id))
return;
var member = await Discord.TryGetGuildMemberAsync(Request.Guild.Id, id, cancellationToken);
// User may have left the guild since they were mentioned
if (member is null)
{
var user = fallbackUser ?? await Discord.TryGetUserAsync(id, cancellationToken);
if (user is not null)
member = Member.CreateDefault(user);
}
// Store the result even if it's null, to avoid re-fetching non-existing members
_members[id] = member;
}
public async ValueTask PopulateMemberAsync(Snowflake id, CancellationToken cancellationToken = default) =>
await PopulateMemberAsync(id, null, cancellationToken);
public async ValueTask PopulateMemberAsync(User user, CancellationToken cancellationToken = default) =>
await PopulateMemberAsync(user.Id, user, cancellationToken);
public string FormatDate(DateTimeOffset instant) => Request.DateFormat switch
{
"unix" => instant.ToUnixTimeSeconds().ToString(),
@@ -93,26 +111,24 @@ internal class ExportContext
try
{
var filePath = await _assetDownloader.DownloadAsync(url, cancellationToken);
var relativeFilePath = Path.GetRelativePath(Request.OutputDirPath, filePath);
// We want relative path so that the output files can be copied around without breaking.
// Base directory path may be null if the file is stored at the root or relative to working directory.
var relativeFilePath = !string.IsNullOrWhiteSpace(Request.OutputBaseDirPath)
? Path.GetRelativePath(Request.OutputBaseDirPath, filePath)
: filePath;
// Prefer relative paths so that the output files can be copied around without breaking references.
// If the assets path is outside of the export directory, use the absolute path instead.
var optimalFilePath =
relativeFilePath.StartsWith(".." + Path.DirectorySeparatorChar, StringComparison.Ordinal) ||
relativeFilePath.StartsWith(".." + Path.AltDirectorySeparatorChar, StringComparison.Ordinal)
? filePath
: relativeFilePath;
// HACK: for HTML, we need to format the URL properly
// For HTML, the path needs to be properly formatted
if (Request.Format is ExportFormat.HtmlDark or ExportFormat.HtmlLight)
{
// Need to escape each path segment while keeping the directory separators intact
return string.Join(
Path.AltDirectorySeparatorChar,
relativeFilePath
.Split(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar)
.Select(Uri.EscapeDataString)
);
// Create a 'file:///' URI and then strip the 'file:///' prefix to allow for relative paths
return new Uri(new Uri("file:///"), optimalFilePath).ToString()[8..];
}
return relativeFilePath;
return optimalFilePath;
}
// Try to catch only exceptions related to failed HTTP requests
// https://github.com/Tyrrrz/DiscordChatExporter/issues/332
@@ -10,36 +10,87 @@ using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Exporting;
public partial record ExportRequest(
Guild Guild,
Channel Channel,
string OutputPath,
ExportFormat Format,
Snowflake? After,
Snowflake? Before,
PartitionLimit PartitionLimit,
MessageFilter MessageFilter,
bool ShouldFormatMarkdown,
bool ShouldDownloadAssets,
bool ShouldReuseAssets,
string DateFormat)
public partial class ExportRequest
{
private string? _outputBaseFilePath;
public string OutputBaseFilePath => _outputBaseFilePath ??= GetOutputBaseFilePath(
Guild,
Channel,
OutputPath,
Format,
After,
Before
);
public Guild Guild { get; }
public string OutputBaseDirPath => Path.GetDirectoryName(OutputBaseFilePath) ?? OutputPath;
public Channel Channel { get; }
public string OutputAssetsDirPath => $"{OutputBaseFilePath}_Files{Path.DirectorySeparatorChar}";
public string OutputFilePath { get; }
public string OutputDirPath { get; }
public string AssetsDirPath { get; }
public ExportFormat Format { get; }
public Snowflake? After { get; }
public Snowflake? Before { get; }
public PartitionLimit PartitionLimit { get; }
public MessageFilter MessageFilter { get; }
public bool ShouldFormatMarkdown { get; }
public bool ShouldDownloadAssets { get; }
public bool ShouldReuseAssets { get; }
public string DateFormat { get; }
public ExportRequest(
Guild guild,
Channel channel,
string outputPath,
string? assetsDirPath,
ExportFormat format,
Snowflake? after,
Snowflake? before,
PartitionLimit partitionLimit,
MessageFilter messageFilter,
bool shouldFormatMarkdown,
bool shouldDownloadAssets,
bool shouldReuseAssets,
string dateFormat)
{
Guild = guild;
Channel = channel;
Format = format;
After = after;
Before = before;
PartitionLimit = partitionLimit;
MessageFilter = messageFilter;
ShouldFormatMarkdown = shouldFormatMarkdown;
ShouldDownloadAssets = shouldDownloadAssets;
ShouldReuseAssets = shouldReuseAssets;
DateFormat = dateFormat;
OutputFilePath = GetOutputBaseFilePath(
Guild,
Channel,
outputPath,
Format,
After,
Before
);
OutputDirPath = Path.GetDirectoryName(OutputFilePath)!;
AssetsDirPath = !string.IsNullOrWhiteSpace(assetsDirPath)
? FormatPath(
assetsDirPath,
Guild,
Channel,
After,
Before
)
: $"{OutputFilePath}_Files{Path.DirectorySeparatorChar}";
}
}
public partial record ExportRequest
public partial class ExportRequest
{
public static string GetDefaultOutputFileName(
Guild guild,
@@ -83,17 +134,15 @@ public partial record ExportRequest
return PathEx.EscapeFileName(buffer.ToString());
}
private static string GetOutputBaseFilePath(
private static string FormatPath(
string path,
Guild guild,
Channel channel,
string outputPath,
ExportFormat format,
Snowflake? after = null,
Snowflake? before = null)
Snowflake? after,
Snowflake? before)
{
// Format path
var actualOutputPath = Regex.Replace(
outputPath,
return Regex.Replace(
path,
"%.",
m => PathEx.EscapeFileName(m.Value switch
{
@@ -112,6 +161,17 @@ public partial record ExportRequest
_ => m.Value
})
);
}
private static string GetOutputBaseFilePath(
Guild guild,
Channel channel,
string outputPath,
ExportFormat format,
Snowflake? after = null,
Snowflake? before = null)
{
var actualOutputPath = FormatPath(outputPath, guild, channel, after, before);
// Output is a directory
if (Directory.Exists(actualOutputPath) || string.IsNullOrWhiteSpace(Path.GetExtension(actualOutputPath)))
@@ -203,7 +203,7 @@ internal partial class HtmlMarkdownVisitor : MarkdownVisitor
var member = mention.TargetId?.Pipe(_context.TryGetMember);
var fullName = member?.User.FullName ?? "Unknown";
var nick = member?.Nick ?? "Unknown";
var nick = member?.Nick ?? member?.User.Name ?? "Unknown";
_buffer.Append(
// language=HTML
@@ -50,7 +50,10 @@ internal class JsonMessageWriter : MessageWriter
_writer.WriteString(
"avatarUrl",
await Context.ResolveAssetUrlAsync(user.AvatarUrl, cancellationToken)
await Context.ResolveAssetUrlAsync(
Context.TryGetMember(user.Id)?.AvatarUrl ?? user.AvatarUrl,
cancellationToken
)
);
_writer.WriteEndObject();
@@ -51,8 +51,8 @@ internal partial class MessageExporter : IAsyncDisposable
if (_writer is not null)
return _writer;
Directory.CreateDirectory(_context.Request.OutputBaseDirPath);
var filePath = GetPartitionFilePath(_context.Request.OutputBaseFilePath, _partitionIndex);
Directory.CreateDirectory(_context.Request.OutputDirPath);
var filePath = GetPartitionFilePath(_context.Request.OutputFilePath, _partitionIndex);
var writer = CreateMessageWriter(filePath, _context.Request.Format, _context);
await writer.WritePreambleAsync(cancellationToken);
@@ -149,7 +149,7 @@
}
// Avatar
<img class="chatlog__avatar" src="@await ResolveAssetUrlAsync(message.Author.AvatarUrl)" alt="Avatar" loading="lazy">
<img class="chatlog__avatar" src="@await ResolveAssetUrlAsync(authorMember?.AvatarUrl ?? message.Author.AvatarUrl)" alt="Avatar" loading="lazy">
}
else
{
@@ -172,7 +172,7 @@
? message.ReferencedMessage.Author.Name
: referencedUserMember?.Nick ?? message.ReferencedMessage.Author.Name;
<img class="chatlog__reply-avatar" src="@await ResolveAssetUrlAsync(message.ReferencedMessage.Author.AvatarUrl)" alt="Avatar" loading="lazy">
<img class="chatlog__reply-avatar" src="@await ResolveAssetUrlAsync(referencedUserMember?.AvatarUrl ?? message.ReferencedMessage.Author.AvatarUrl)" alt="Avatar" loading="lazy">
<div class="chatlog__reply-author" style="@(referencedUserColor is not null ? $"color: rgb({referencedUserColor.Value.R}, {referencedUserColor.Value.G}, {referencedUserColor.Value.B})" : null)" title="@message.ReferencedMessage.Author.FullName">@referencedUserNick</div>
<div class="chatlog__reply-content">
<span class="chatlog__reply-link" onclick="scrollToMessage(event, '@message.ReferencedMessage.Id')">
@@ -205,7 +205,7 @@
? message.Interaction.User.Name
: interactionUserMember?.Nick ?? message.Interaction.User.Name;
<img class="chatlog__reply-avatar" src="@await ResolveAssetUrlAsync(message.Interaction.User.AvatarUrl)" alt="Avatar" loading="lazy">
<img class="chatlog__reply-avatar" src="@await ResolveAssetUrlAsync(interactionUserMember?.AvatarUrl ?? message.Interaction.User.AvatarUrl)" alt="Avatar" loading="lazy">
<div class="chatlog__reply-author" style="@(interactionUserColor is not null ? $"color: rgb({interactionUserColor.Value.R}, {interactionUserColor.Value.G}, {interactionUserColor.Value.B})" : null)" title="@message.Interaction.User.FullName">@interactionUserNick</div>
<div class="chatlog__reply-content">
used /@message.Interaction.Name
@@ -329,22 +329,22 @@
<div class="chatlog__embed">
<div class="chatlog__embed-invite-container">
<div class="chatlog__embed-invite-title">Invite to join a server</div>
<div class="chatlog__embed-invite-title">@(invite.Channel?.Kind.IsDirect() == true ? "Invite to join a group DM" : "Invite to join a server")</div>
<div class="chatlog__embed-invite">
<div class="chatlog__embed-invite-guild-icon-container">
<img class="chatlog__embed-invite-guild-icon" src="@await ResolveAssetUrlAsync(invite.Guild?.IconUrl ?? Guild.DirectMessages.IconUrl)" alt="Guild icon" loading="lazy">
<img class="chatlog__embed-invite-guild-icon" src="@await ResolveAssetUrlAsync(invite.Channel?.IconUrl ?? invite.Guild.IconUrl)" alt="Guild icon" loading="lazy">
</div>
<div class="chatlog__embed-invite-info">
<div class="chatlog__embed-invite-guild-name">
<a href="https://discord.gg/@invite.Code">
@(invite.Guild?.Name ?? "Unknown guild")
@(invite.Guild.Name)
</a>
</div>
<div class="chatlog__embed-invite-channel-name">
<svg class="chatlog__embed-invite-channel-icon">
<use href="#channel-icon"></use>
</svg>
<span> @(invite.Channel?.Name ?? "Unknown channel")</span>
<span> @(invite.Channel?.Name ?? "Unknown Channel")</span>
</div>
</div>
</div>
@@ -447,6 +447,19 @@
</a>
</div>
}
// Generic video embed
else if (embed.Kind == EmbedKind.Video && !string.IsNullOrWhiteSpace(embed.Url))
{
var embedVideoUrl =
embed.Video?.ProxyUrl ?? embed.Video?.Url ??
embed.Url;
<div class="chatlog__embed">
<video class="chatlog__embed-generic-video" width="@embed.Video?.Width" height="@embed.Video?.Height" controls>
<source src="@await ResolveAssetUrlAsync(embedVideoUrl)" alt="Embedded video">
</video>
</div>
}
// Generic gifv embed
else if (embed.Kind == EmbedKind.Gifv && !string.IsNullOrWhiteSpace(embed.Url))
{
@@ -455,8 +468,8 @@
embed.Url;
<div class="chatlog__embed">
<video class="chatlog__embed-generic-gifv" loop width="@embed.Video?.Width" height="@embed.Video?.Height" onmouseover="this.play()" onmouseout="this.pause()">
<source src="@await ResolveAssetUrlAsync(embedVideoUrl)" alt="Embedded video">
<video class="chatlog__embed-generic-gifv" width="@embed.Video?.Width" height="@embed.Video?.Height" loop onmouseover="this.play()" onmouseout="this.pause()">
<source src="@await ResolveAssetUrlAsync(embedVideoUrl)" alt="Embedded gifv">
</video>
</div>
}
@@ -621,7 +634,7 @@
@foreach (var sticker in message.Stickers)
{
<div class="chatlog__sticker" title="@sticker.Name">
@if (sticker.Format is StickerFormat.Png or StickerFormat.PngAnimated)
@if (sticker.Format is StickerFormat.Png or StickerFormat.Apng)
{
<img class="chatlog__sticker--media" src="@await ResolveAssetUrlAsync(sticker.SourceUrl)" alt="Sticker">
}
@@ -15,8 +15,8 @@
? darkVariant
: lightVariant;
string GetFontUrl(int weight) =>
$"https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-{weight}.woff";
string GetFontUrl(string style, int weight) =>
$"https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/ggsans-{style}-{weight}.woff2";
ValueTask<string> ResolveAssetUrlAsync(string url) =>
Context.ResolveAssetUrlAsync(url, CancellationToken);
@@ -41,33 +41,73 @@
@{/* Styling */}
<style>
@@font-face {
src: url(@await ResolveAssetUrlAsync(GetFontUrl(300)));
font-family: Whitney;
font-weight: 300;
}
@@font-face {
src: url(@await ResolveAssetUrlAsync(GetFontUrl(400)));
font-family: Whitney;
src: url("@await ResolveAssetUrlAsync(GetFontUrl("normal", 400))");
font-family: gg sans;
font-weight: 400;
font-style: normal;
}
@@font-face {
src: url(@await ResolveAssetUrlAsync(GetFontUrl(500)));
font-family: Whitney;
src: url("@await ResolveAssetUrlAsync(GetFontUrl("normal", 500))");
font-family: gg sans;
font-weight: 500;
font-style: normal;
}
@@font-face {
src: url(@await ResolveAssetUrlAsync(GetFontUrl(600)));
font-family: Whitney;
src: url("@await ResolveAssetUrlAsync(GetFontUrl("normal", 600))");
font-family: gg sans;
font-weight: 600;
font-style: normal;
}
@@font-face {
src: url(@await ResolveAssetUrlAsync(GetFontUrl(700)));
font-family: Whitney;
src: url("@await ResolveAssetUrlAsync(GetFontUrl("normal", 700))");
font-family: gg sans;
font-weight: 700;
font-style: normal;
}
@@font-face {
src: url("@await ResolveAssetUrlAsync(GetFontUrl("normal", 800))");
font-family: gg sans;
font-weight: 800;
font-style: normal;
}
@@font-face {
src: url("@await ResolveAssetUrlAsync(GetFontUrl("italic", 400))");
font-family: gg sans;
font-weight: 400;
font-style: italic;
}
@@font-face {
src: url("@await ResolveAssetUrlAsync(GetFontUrl("italic", 500))");
font-family: gg sans;
font-weight: 500;
font-style: italic;
}
@@font-face {
src: url("@await ResolveAssetUrlAsync(GetFontUrl("italic", 600))");
font-family: gg sans;
font-weight: 600;
font-style: italic;
}
@@font-face {
src: url("@await ResolveAssetUrlAsync(GetFontUrl("italic", 700))");
font-family: gg sans;
font-weight: 700;
font-style: italic;
}
@@font-face {
src: url("@await ResolveAssetUrlAsync(GetFontUrl("italic", 800))");
font-family: gg sans;
font-weight: 800;
font-style: italic;
}
html, body {
@@ -75,7 +115,7 @@
padding: 0;
background-color: @Themed("#36393e", "#ffffff");
color: @Themed("#dcddde", "#23262a");
font-family: Whitney, "Helvetica Neue", Helvetica, Arial, sans-serif;
font-family: "gg sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
font-size: 17px;
font-weight: @Themed("400", "500");
scroll-behavior: smooth;
@@ -582,24 +622,6 @@
font-weight: 500;
}
.chatlog__embed-generic-image {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-generic-gifv {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-invite-container {
min-width: 320px;
padding: 0.6rem 0.7rem;
@@ -650,6 +672,33 @@
font-weight: 600;
}
.chatlog__embed-generic-image {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-generic-video {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-generic-gifv {
object-fit: contain;
object-position: left;
max-width: 45vw;
max-height: 500px;
vertical-align: top;
border-radius: 3px;
}
.chatlog__embed-spotify {
border: 0;
}
@@ -1,40 +0,0 @@
using System;
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Utils;
public static class FileFormat
{
private static readonly HashSet<string> ImageExtensions = new(StringComparer.OrdinalIgnoreCase)
{
".jpg",
".jpeg",
".png",
".gif",
".gifv",
".bmp",
".webp"
};
public static bool IsImage(string format) => ImageExtensions.Contains(format);
private static readonly HashSet<string> VideoExtensions = new(StringComparer.OrdinalIgnoreCase)
{
".mp4",
".webm",
".mov"
};
public static bool IsVideo(string format) => VideoExtensions.Contains(format);
private static readonly HashSet<string> AudioExtensions = new(StringComparer.OrdinalIgnoreCase)
{
".mp3",
".wav",
".ogg",
".flac",
".m4a"
};
public static bool IsAudio(string format) => AudioExtensions.Contains(format);
}
@@ -13,15 +13,15 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="Cogwheel" Version="2.0.1" />
<PackageReference Include="FuckRussia" Version="1.0.1" PrivateAssets="all" />
<PackageReference Include="Gress" Version="2.0.1" />
<PackageReference Include="MaterialDesignColors" Version="2.1.1" />
<PackageReference Include="MaterialDesignThemes" Version="4.7.1" />
<PackageReference Include="Microsoft.Xaml.Behaviors.Wpf" Version="1.1.39" />
<PackageReference Include="Ookii.Dialogs.Wpf" Version="5.0.1" />
<PackageReference Include="Onova" Version="2.6.4" />
<PackageReference Include="Onova" Version="2.6.6" />
<PackageReference Include="Stylet" Version="1.3.6" />
<PackageReference Include="Tyrrrz.Settings" Version="1.3.4" />
<PackageReference Include="PropertyChanged.Fody" Version="4.1.0" PrivateAssets="all" />
<PackageReference Include="DotnetRuntimeBootstrapper" Version="2.4.0" PrivateAssets="all" />
</ItemGroup>
@@ -1,11 +1,12 @@
using System;
using System.IO;
using Cogwheel;
using DiscordChatExporter.Core.Exporting;
using Microsoft.Win32;
using Tyrrrz.Settings;
namespace DiscordChatExporter.Gui.Services;
public partial class SettingsService : SettingsManager
public partial class SettingsService : SettingsBase
{
public bool IsUkraineSupportMessageEnabled { get; set; } = true;
@@ -35,14 +36,21 @@ public partial class SettingsService : SettingsManager
public bool LastShouldReuseAssets { get; set; }
public string? LastAssetsDirPath { get; set; }
public SettingsService()
: base(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Settings.dat"))
{
Configuration.StorageSpace = StorageSpace.Instance;
Configuration.SubDirectoryPath = "";
Configuration.FileName = "Settings.dat";
}
public bool ShouldSerializeLastToken() => IsTokenPersisted;
public override void Save()
{
// Clear token if it's not supposed to be persisted
if (!IsTokenPersisted)
LastToken = null;
base.Save();
}
}
public partial class SettingsService
@@ -38,16 +38,16 @@ public class DashboardViewModel : PropertyChangedBase
public string? Token { get; set; }
private IReadOnlyDictionary<Guild, IReadOnlyList<Channel>>? GuildChannelMap { get; set; }
private IReadOnlyDictionary<Guild, IReadOnlyList<Channel>>? ChannelsByGuild { get; set; }
public IReadOnlyList<Guild>? AvailableGuilds => GuildChannelMap?.Keys.ToArray();
public IReadOnlyList<Guild>? AvailableGuilds => ChannelsByGuild?.Keys.ToArray();
public Guild? SelectedGuild { get; set; }
public bool IsDirectMessageGuildSelected => SelectedGuild?.Id == Guild.DirectMessages.Id;
public IReadOnlyList<Channel>? AvailableChannels => SelectedGuild is not null
? GuildChannelMap?[SelectedGuild]
? ChannelsByGuild?[SelectedGuild]
: null;
public IReadOnlyList<Channel>? SelectedChannels { get; set; }
@@ -71,10 +71,8 @@ public class DashboardViewModel : PropertyChangedBase
public void OnViewLoaded()
{
if (_settingsService.LastToken is not null)
{
if (!string.IsNullOrWhiteSpace(_settingsService.LastToken))
Token = _settingsService.LastToken;
}
}
public async void ShowSettings()
@@ -103,17 +101,17 @@ public class DashboardViewModel : PropertyChangedBase
var discord = new DiscordClient(token);
var guildChannelMap = new Dictionary<Guild, IReadOnlyList<Channel>>();
var channelsByGuild = new Dictionary<Guild, IReadOnlyList<Channel>>();
await foreach (var guild in discord.GetUserGuildsAsync())
{
guildChannelMap[guild] = (await discord.GetGuildChannelsAsync(guild.Id))
channelsByGuild[guild] = (await discord.GetGuildChannelsAsync(guild.Id))
.Where(c => c.Kind != ChannelKind.GuildCategory)
.ToArray();
}
_discord = discord;
GuildChannelMap = guildChannelMap;
SelectedGuild = guildChannelMap.Keys.FirstOrDefault();
ChannelsByGuild = channelsByGuild;
SelectedGuild = channelsByGuild.Keys.FirstOrDefault();
}
catch (DiscordChatExporterException ex) when (!ex.IsFatal)
{
@@ -186,6 +184,7 @@ public class DashboardViewModel : PropertyChangedBase
dialog.Guild!,
channel,
dialog.OutputPath!,
dialog.AssetsDirPath,
dialog.SelectedFormat,
dialog.After?.Pipe(Snowflake.FromDate),
dialog.Before?.Pipe(Snowflake.FromDate),
@@ -65,6 +65,8 @@ public class ExportSetupViewModel : DialogScreen
public bool ShouldReuseAssets { get; set; }
public string? AssetsDirPath { get; set; }
public bool IsAdvancedSectionDisplayed { get; set; }
public ExportSetupViewModel(DialogManager dialogManager, SettingsService settingsService)
@@ -79,15 +81,18 @@ public class ExportSetupViewModel : DialogScreen
ShouldFormatMarkdown = _settingsService.LastShouldFormatMarkdown;
ShouldDownloadAssets = _settingsService.LastShouldDownloadAssets;
ShouldReuseAssets = _settingsService.LastShouldReuseAssets;
AssetsDirPath = _settingsService.LastAssetsDirPath;
// Show the "advanced options" section by default if any
// of the advanced options are set to non-default values.
IsAdvancedSectionDisplayed =
After != default ||
Before != default ||
After is not null ||
Before is not null ||
!string.IsNullOrWhiteSpace(PartitionLimitValue) ||
!string.IsNullOrWhiteSpace(MessageFilterValue) ||
ShouldDownloadAssets != default;
ShouldDownloadAssets ||
ShouldReuseAssets ||
!string.IsNullOrWhiteSpace(AssetsDirPath);
}
public void ToggleAdvancedSection() => IsAdvancedSectionDisplayed = !IsAdvancedSectionDisplayed;
@@ -107,18 +112,25 @@ public class ExportSetupViewModel : DialogScreen
var extension = SelectedFormat.GetFileExtension();
var filter = $"{extension.ToUpperInvariant()} files|*.{extension}";
var outputPath = _dialogManager.PromptSaveFilePath(filter, defaultFileName);
if (!string.IsNullOrWhiteSpace(outputPath))
OutputPath = outputPath;
var path = _dialogManager.PromptSaveFilePath(filter, defaultFileName);
if (!string.IsNullOrWhiteSpace(path))
OutputPath = path;
}
else
{
var outputPath = _dialogManager.PromptDirectoryPath();
if (!string.IsNullOrWhiteSpace(outputPath))
OutputPath = outputPath;
var path = _dialogManager.PromptDirectoryPath();
if (!string.IsNullOrWhiteSpace(path))
OutputPath = path;
}
}
public void ShowAssetsDirPathPrompt()
{
var path = _dialogManager.PromptDirectoryPath();
if (!string.IsNullOrWhiteSpace(path))
AssetsDirPath = path;
}
public void Confirm()
{
// Prompt the output path if it's not set yet
@@ -138,6 +150,7 @@ public class ExportSetupViewModel : DialogScreen
_settingsService.LastShouldFormatMarkdown = ShouldFormatMarkdown;
_settingsService.LastShouldDownloadAssets = ShouldDownloadAssets;
_settingsService.LastShouldReuseAssets = ShouldReuseAssets;
_settingsService.LastAssetsDirPath = AssetsDirPath;
Close(true);
}
@@ -145,8 +158,10 @@ public class ExportSetupViewModel : DialogScreen
public static class ExportSetupViewModelExtensions
{
public static ExportSetupViewModel CreateExportSetupViewModel(this IViewModelFactory factory,
Guild guild, IReadOnlyList<Channel> channels)
public static ExportSetupViewModel CreateExportSetupViewModel(
this IViewModelFactory factory,
Guild guild,
IReadOnlyList<Channel> channels)
{
var viewModel = factory.CreateExportSetupViewModel();
@@ -25,8 +25,10 @@ public static class MessageBoxViewModelExtensions
{
public static MessageBoxViewModel CreateMessageBoxViewModel(
this IViewModelFactory factory,
string title, string message,
string? okButtonText, string? cancelButtonText)
string title,
string message,
string? okButtonText,
string? cancelButtonText)
{
var viewModel = factory.CreateMessageBoxViewModel();
viewModel.Title = title;
@@ -42,6 +44,7 @@ public static class MessageBoxViewModelExtensions
public static MessageBoxViewModel CreateMessageBoxViewModel(
this IViewModelFactory factory,
string title, string message) =>
string title,
string message) =>
factory.CreateMessageBoxViewModel(title, message, "CLOSE", null);
}
@@ -19,7 +19,7 @@
mc:Ignorable="d">
<UserControl.Resources>
<!-- Sort DMs by last message -->
<CollectionViewSource x:Key="AvailableDirectMessageChannelsViewSource" Source="{Binding AvailableChannels, Mode=OneWay}">
<CollectionViewSource x:Key="AvailableDirectChannelsViewSource" Source="{Binding AvailableChannels, Mode=OneWay}">
<CollectionViewSource.SortDescriptions>
<componentModel:SortDescription Direction="Descending" PropertyName="LastMessageId" />
<componentModel:SortDescription Direction="Ascending" PropertyName="Name" />
@@ -300,7 +300,7 @@
<Style BasedOn="{StaticResource {x:Type ListBox}}" TargetType="{x:Type ListBox}">
<Style.Triggers>
<DataTrigger Binding="{Binding IsDirectMessageGuildSelected}" Value="True">
<Setter Property="ItemsSource" Value="{Binding Source={StaticResource AvailableDirectMessageChannelsViewSource}}" />
<Setter Property="ItemsSource" Value="{Binding Source={StaticResource AvailableDirectChannelsViewSource}}" />
</DataTrigger>
<DataTrigger Binding="{Binding IsDirectMessageGuildSelected}" Value="False">
<Setter Property="ItemsSource" Value="{Binding Source={StaticResource AvailableChannelsViewSource}}" />
@@ -285,6 +285,27 @@
VerticalAlignment="Center"
IsChecked="{Binding ShouldReuseAssets}" />
</Grid>
<!-- Assets path -->
<Grid Margin="16,8">
<TextBox
Padding="16,16,42,16"
materialDesign:HintAssist.Hint="Assets directory path"
materialDesign:HintAssist.IsFloating="True"
Style="{DynamicResource MaterialDesignOutlinedTextBox}"
Text="{Binding AssetsDirPath}"
ToolTip="Download assets to this directory. If not specified, the asset directory path will be derived from the output path." />
<Button
Width="24"
Height="24"
Margin="0,0,12,0"
Padding="0"
HorizontalAlignment="Right"
Command="{s:Action ShowAssetsDirPathPrompt}"
Style="{DynamicResource MaterialDesignToolForegroundButton}">
<materialDesign:PackIcon Kind="FolderOpen" />
</Button>
</Grid>
</StackPanel>
</StackPanel>
</ScrollViewer>
+1 -1
View File
@@ -27,7 +27,7 @@ By using this project or its source code, for any purpose and in any shape or fo
- You **support Ukraine's territorial integrity, including its claims over temporarily occupied territories of Crimea and Donbas**
- You **reject false narratives perpetuated by Russian state propaganda**
To learn more about the war and how you can help, [click here](https://tyrrrz.me). Glory to Ukraine! 🇺🇦
To learn more about the war and how you can help, [click here](https://tyrrrz.me/ukraine). Glory to Ukraine! 🇺🇦
## Download