Compare commits

..

29 Commits

Author SHA1 Message Date
Tyrrrz cba5a934df Update version 2021-06-19 20:50:22 +03:00
Tyrrrz cd05058cf7 Prefer proxy URLs for embed content everywhere 2021-06-19 20:25:50 +03:00
Tyrrrz e9c3872701 [HTML] Prevent click events on elements inside referenced message
Closes #545
2021-06-19 20:17:36 +03:00
Tyrrrz 88c4330a66 [HTML] Link to pre-resized avatar image for better image quality 2021-06-19 20:15:18 +03:00
Tyrrrz 998bdaf2d4 [HTML] Use svg variants of Twemoji for better resize quality 2021-06-19 19:57:29 +03:00
Tyrrrz 24a80f915f [HTML] Recognize standard emoji by code and show emoji code in tooltips
Closes #549
Closes #599
2021-06-19 19:49:53 +03:00
Tyrrrz de57cd714d [HTML] Prepend "Message sent: " to message timestamp tooltips 2021-06-19 18:56:47 +03:00
Tyrrrz f9ab269b29 [HTML] Don't jumbo emojis in referenced messages
Closes #544
2021-06-19 15:49:16 +03:00
Tyrrrz 33feaccee7 Only use DotnetRuntimeBootstrapper in release build 2021-06-17 22:08:11 +03:00
Tyrrrz 4fd6f8272e [HTML] Update styling to match Discord 2021-06-16 18:15:58 +03:00
Tyrrrz b28f029382 Refactor YouTube embeds 2021-06-16 17:54:35 +03:00
Tyrrrz 29552be6e5 Cleanup 2021-06-16 16:52:18 +03:00
Tyrrrz 9491e18e2f Update NuGet packages 2021-06-15 22:41:06 +03:00
quentinmay 4b1c9660be Render YouTube iframe for applicable video embeds (#607) 2021-06-15 07:19:19 -07:00
Tyrrrz 950625e671 [HTML] Show timestamps of individual messages
Closes #536
2021-06-14 22:55:42 +03:00
Tyrrrz ce2dedff6b [HTML] Use a clickable <div> instead of <a> for reply reference link
Fixes #546
2021-06-14 21:33:20 +03:00
Tyrrrz baac419538 [HTML] Fix embed footer styling 2021-06-14 20:48:20 +03:00
Tyrrrz bd40dbde48 Don't throw if file dates cannot be modified when downloading media
Fixes #585
2021-06-14 20:43:44 +03:00
Tyrrrz cbe4bc593c [HTML] Prefer proxy URLs for resolving embed media
Closes #590
2021-06-14 20:38:19 +03:00
Tyrrrz cb32cee5f5 Update DotnetRuntimeBootstrapper 2021-06-14 04:51:32 +03:00
Tyrrrz 82f364a887 Update readme 2021-06-08 18:04:19 +03:00
Tyrrrz 152e8c8dcb Update version 2021-06-08 17:54:58 +03:00
John Zabroski 260139ab25 Fix .NET v3.1 to say .NET Core v3.1 in readme (#605) 2021-06-08 07:25:12 -07:00
Tyrrrz 11e9c0f15c Downgrade MDIX 2021-06-08 01:32:21 +03:00
Tyrrrz ac809d1a3f Update packages 2021-06-08 01:23:00 +03:00
Tyrrrz 36fa346299 Use DotnetRuntimeBootstrapper 2021-06-08 00:53:41 +03:00
Tyrrrz 8c3a5bd096 Update issue templates 2021-05-12 21:38:12 +03:00
Tyrrrz 021682899a [CLI] Move Markup.Escape(...) to ConsoleExtensions 2021-04-26 19:47:25 +03:00
Lucas LaBuff 0ec6027a93 Escape brackets in category name in CLI (#558) 2021-04-26 09:37:37 -07:00
51 changed files with 9322 additions and 276 deletions
+14 -1
View File
@@ -5,9 +5,11 @@ body:
- type: markdown
attributes:
value: |
Important:
- Please check existing issues (both opened and closed) to ensure that this bug hasn't been reported before.
- Refer to the [contribution guidelines](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/Contributing.md) for info on how to file a good issue.
- If you want to ask a question instead of reporting a bug, use [**discussions**](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
- If you want to ask a question instead of reporting a bug, please use [discussions](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
- type: input
attributes:
@@ -27,6 +29,17 @@ body:
validations:
required: true
- type: dropdown
attributes:
label: Export format
description: "Which export format(s) does this bug affect? If this bug is not related to any specific export format, leave this blank."
multiple: true
options:
- HTML
- TXT
- JSON
- CSV
- type: textarea
attributes:
label: Details
+16 -3
View File
@@ -5,22 +5,35 @@ body:
- type: markdown
attributes:
value: |
Important:
- Please check existing issues (both opened and closed) to ensure that this feature hasn't been requested before.
- Refer to the [contribution guidelines](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/Contributing.md) for info on how to file a good issue.
- If you want to ask a question instead of requesting a feature, use [**discussions**](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
- If you want to ask a question instead of requesting a feature, please use [discussions](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
- type: dropdown
attributes:
label: Flavor
description: "Which flavor(s) of DiscordChatExporter does this feature request apply to?"
description: "Which flavor(s) of DiscordChatExporter does this feature request apply to? If this feature is not related to any specific flavor, leave this blank."
multiple: true
options:
- GUI (Graphical User Interface)
- CLI (Command Line Interface)
- type: dropdown
attributes:
label: Export format
description: "Which export format(s) does this feature request apply to? If this feature is not related to any specific export format, leave this blank."
multiple: true
options:
- HTML
- TXT
- JSON
- CSV
- type: textarea
attributes:
label: Details
description: "Clear and thorough explanation of the feature you have in mind."
description: "Clear and thorough explanation of the feature you have in mind. If relevant, include screenshots or screen recordings."
validations:
required: true
+23
View File
@@ -1,3 +1,26 @@
### v2.28 (19-Jun-2021)
- [HTML] Added special casing for YouTube video embeds to render them directly using an embedded player. (Thanks [@quentinmay](https://github.com/quentinmay))
- [HTML] Added support for rendering standard emoji by code. Such emoji may sometimes be found in messages sent by bots or through webhooks. (Thanks [@CanePlayz](https://github.com/CanePlayz) and [@Lucas LaBuff](https://github.com/96-LB))
- [HTML] Changed tooltips on standard emoji to show emoji code instead of their raw string representation. (Thanks [@CanePlayz](https://github.com/CanePlayz) and [@Lucas LaBuff](https://github.com/96-LB))
- [HTML] Added tooltips on individual messages to show when those messages were sent.
- [HTML] Updated colors and minor styling elements to match Discord's new style direction after rebranding.
- [HTML] Updated Twemoji URLs to use SVG image variants for better rendering quality.
- [HTML] Changed user avatar URLs to include predetermined size query parameter for better rendering quality.
- Changed embed rendering logic to prefer Discord-proxied URLs for external content, in order to avoid unnecessary HTTP requests to third parties.
- Fixed an issue where exporting with media sometimes resulted in a crash due to lack of permissions to change file attributes. These errors are now ignored, which means that the creation, write, and access dates of downloaded files may sometimes not match the dates provided by Discord's CDN.
- [HTML] Fixed styling issues when rendering embed footers.
- [HTML] Fixed an issue where links didn't render properly inside referenced messages.
- [HTML] Fixed an issue where standalone emoji were incorrectly enlarged inside referenced messages.
- [HTML] Fixed an issue where clicking on a referenced message revealed spoilers inside of it.
- [GUI] Fixed an issue where the list of missing components, reported by the prerequisite check at application startup, included Windows updates that were not applicable to the user's system due to the fact that other superseding updates have already been installed. This issue only affected users running DiscordChatExporter on Windows 7.
- [GUI] Fixed a few other minor issues related to the prerequisite installation window.
### v2.27.1 (08-Jun-2021)
- [GUI] Application will now detect if the required .NET Runtime or any of its prerequisites are missing and prompt the user to download and install them automatically. **Experimental feature, please test it out and report any issues you may find!**
- [CLI] Fixed an issue where the application crashed when exporting a channel that had square brackets as part of its name or category name. (Thanks [@Lucas LaBuff](https://github.com/96-LB))
### v2.27 (24-Apr-2021)
- Added partitioning by file size. You can now use values such as `10mb` to indicate a size-based cut off point, in addition to values like `10` to indicate a number of messages. (Thanks [@Andrew Kolos](https://github.com/andrewkolos))
+2 -2
View File
@@ -2,7 +2,7 @@
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<Version>2.27</Version>
<Version>2.28</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Alexey Golub</Copyright>
<LangVersion>preview</LangVersion>
@@ -10,4 +10,4 @@
<WarningsAsErrors>nullable</WarningsAsErrors>
</PropertyGroup>
</Project>
</Project>
@@ -7,7 +7,7 @@
<ItemGroup>
<PackageReference Include="CliFx" Version="2.0.4" />
<PackageReference Include="Spectre.Console" Version="0.39.0" />
<PackageReference Include="Spectre.Console" Version="0.40.0" />
<PackageReference Include="Gress" Version="1.2.0" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.5" />
</ItemGroup>
@@ -33,7 +33,11 @@ namespace DiscordChatExporter.Cli.Utils.Extensions
string description,
Func<ProgressTask, ValueTask> performOperationAsync)
{
var progressTask = progressContext.AddTask(description, new ProgressTaskSettings {MaxValue = 1});
var progressTask = progressContext.AddTask(
// Don't recognize random square brackets as style tags
Markup.Escape(description),
new ProgressTaskSettings {MaxValue = 1}
);
try
{
@@ -2,8 +2,6 @@
namespace DiscordChatExporter.Core.Discord
{
public enum AuthTokenType { User, Bot }
public class AuthToken
{
public AuthTokenType Type { get; }
@@ -0,0 +1,8 @@
namespace DiscordChatExporter.Core.Discord
{
public enum AuthTokenType
{
User,
Bot
}
}
@@ -7,32 +7,19 @@ using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Discord.Data
{
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
// Order of enum fields needs to match the order in the docs.
public enum ChannelType
{
GuildTextChat,
DirectTextChat,
GuildVoiceChat,
DirectGroupTextChat,
GuildCategory,
GuildNews,
GuildStore
}
// https://discord.com/developers/docs/resources/channel#channel-object
public partial class Channel : IHasId, IHasPosition
public partial class Channel : IHasId
{
public Snowflake Id { get; }
public ChannelType Type { get; }
public bool IsTextChannel =>
Type == ChannelType.GuildTextChat ||
Type == ChannelType.DirectTextChat ||
Type == ChannelType.DirectGroupTextChat ||
Type == ChannelType.GuildNews ||
Type == ChannelType.GuildStore;
public bool IsTextChannel => Type is
ChannelType.GuildTextChat or
ChannelType.DirectTextChat or
ChannelType.DirectGroupTextChat or
ChannelType.GuildNews or
ChannelType.GuildStore;
public Snowflake GuildId { get; }
@@ -48,7 +35,7 @@ namespace DiscordChatExporter.Core.Discord.Data
Snowflake id,
ChannelType type,
Snowflake guildId,
ChannelCategory? category,
ChannelCategory category,
string name,
int? position,
string? topic)
@@ -56,14 +43,13 @@ namespace DiscordChatExporter.Core.Discord.Data
Id = id;
Type = type;
GuildId = guildId;
Category = category ?? GetFallbackCategory(type);
Category = category;
Name = name;
Position = position;
Topic = topic;
}
public override string ToString() => Name;
}
public partial class Channel
@@ -79,7 +65,7 @@ namespace DiscordChatExporter.Core.Discord.Data
ChannelType.GuildStore => "Store",
_ => "Default"
},
0
null
);
public static Channel Parse(JsonElement json, ChannelCategory? category = null, int? position = null)
@@ -87,23 +73,23 @@ namespace DiscordChatExporter.Core.Discord.Data
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);
var guildId = json.GetPropertyOrNull("guild_id")?.GetString().Pipe(Snowflake.Parse);
var topic = json.GetPropertyOrNull("topic")?.GetString();
var type = (ChannelType) json.GetProperty("type").GetInt32();
var name =
// Guild channel
json.GetPropertyOrNull("name")?.GetString() ??
// DM channel
json.GetPropertyOrNull("recipients")?.EnumerateArray().Select(User.Parse).Select(u => u.Name).JoinToString(", ") ??
// Fallback
id.ToString();
position ??= json.GetPropertyOrNull("position")?.GetInt32();
return new Channel(
id,
type,
guildId ?? Guild.DirectMessages.Id,
category ?? GetFallbackCategory(type),
name,
position,
position ?? json.GetPropertyOrNull("position")?.GetInt32(),
topic
);
}
@@ -1,13 +1,11 @@
using System.Linq;
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Discord.Data
{
public partial class ChannelCategory : IHasId, IHasPosition
public partial class ChannelCategory : IHasId
{
public Snowflake Id { get; }
@@ -23,7 +21,6 @@ namespace DiscordChatExporter.Core.Discord.Data
}
public override string ToString() => Name;
}
public partial class ChannelCategory
@@ -31,19 +28,18 @@ namespace DiscordChatExporter.Core.Discord.Data
public static ChannelCategory Parse(JsonElement json, int? position = null)
{
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);
position ??= json.GetPropertyOrNull("position")?.GetInt32();
var name = json.GetPropertyOrNull("name")?.GetString() ??
json.GetPropertyOrNull("recipients")?.EnumerateArray().Select(User.Parse).Select(u => u.Name).JoinToString(", ") ??
var name =
json.GetPropertyOrNull("name")?.GetString() ??
id.ToString();
return new ChannelCategory(
id,
name,
position
position ?? json.GetPropertyOrNull("position")?.GetInt32()
);
}
public static ChannelCategory Empty { get; } = new(Snowflake.Zero, "<unknown category>", 0);
}
}
}
@@ -0,0 +1,15 @@
namespace DiscordChatExporter.Core.Discord.Data
{
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
// Order of enum fields needs to match the order in the docs.
public enum ChannelType
{
GuildTextChat = 0,
DirectTextChat,
GuildVoiceChat,
DirectGroupTextChat,
GuildCategory,
GuildNews,
GuildStore
}
}
@@ -1,7 +0,0 @@
namespace DiscordChatExporter.Core.Discord.Data.Common
{
public interface IHasPosition
{
int? Position { get; }
}
}
@@ -55,6 +55,8 @@ namespace DiscordChatExporter.Core.Discord.Data
Footer = footer;
}
public YouTubeVideoEmbedProjection? TryGetYouTubeVideo() => YouTubeVideoEmbedProjection.TryResolve(this);
public override string ToString() => Title ?? "<untitled embed>";
}
@@ -63,10 +65,10 @@ namespace DiscordChatExporter.Core.Discord.Data
public static Embed Parse(JsonElement json)
{
var title = json.GetPropertyOrNull("title")?.GetString();
var description = json.GetPropertyOrNull("description")?.GetString();
var url = json.GetPropertyOrNull("url")?.GetString();
var timestamp = json.GetPropertyOrNull("timestamp")?.GetDateTimeOffset();
var color = json.GetPropertyOrNull("color")?.GetInt32().Pipe(System.Drawing.Color.FromArgb).ResetAlpha();
var description = json.GetPropertyOrNull("description")?.GetString();
var author = json.GetPropertyOrNull("author")?.Pipe(EmbedAuthor.Parse);
var thumbnail = json.GetPropertyOrNull("thumbnail")?.Pipe(EmbedImage.Parse);
@@ -12,11 +12,14 @@ namespace DiscordChatExporter.Core.Discord.Data
public string? IconUrl { get; }
public EmbedAuthor(string? name, string? url, string? iconUrl)
public string? IconProxyUrl { get; }
public EmbedAuthor(string? name, string? url, string? iconUrl, string? iconProxyUrl)
{
Name = name;
Url = url;
IconUrl = iconUrl;
IconProxyUrl = iconProxyUrl;
}
public override string ToString() => Name ?? "<unnamed author>";
@@ -29,8 +32,9 @@ namespace DiscordChatExporter.Core.Discord.Data
var name = json.GetPropertyOrNull("name")?.GetString();
var url = json.GetPropertyOrNull("url")?.GetString();
var iconUrl = json.GetPropertyOrNull("icon_url")?.GetString();
var iconProxyUrl = json.GetPropertyOrNull("proxy_icon_url")?.GetString();
return new EmbedAuthor(name, url, iconUrl);
return new EmbedAuthor(name, url, iconUrl, iconProxyUrl);
}
}
}
@@ -10,10 +10,13 @@ namespace DiscordChatExporter.Core.Discord.Data
public string? IconUrl { get; }
public EmbedFooter(string text, string? iconUrl)
public string? IconProxyUrl { get; }
public EmbedFooter(string text, string? iconUrl, string? iconProxyUrl)
{
Text = text;
IconUrl = iconUrl;
IconProxyUrl = iconProxyUrl;
}
public override string ToString() => Text;
@@ -25,8 +28,9 @@ namespace DiscordChatExporter.Core.Discord.Data
{
var text = json.GetProperty("text").GetString();
var iconUrl = json.GetPropertyOrNull("icon_url")?.GetString();
var iconProxyUrl = json.GetPropertyOrNull("proxy_icon_url")?.GetString();
return new EmbedFooter(text, iconUrl);
return new EmbedFooter(text, iconUrl, iconProxyUrl);
}
}
}
@@ -8,13 +8,16 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public string? Url { get; }
public string? ProxyUrl { get; }
public int? Width { get; }
public int? Height { get; }
public EmbedImage(string? url, int? width, int? height)
public EmbedImage(string? url, string? proxyUrl, int? width, int? height)
{
Url = url;
ProxyUrl = proxyUrl;
Height = height;
Width = width;
}
@@ -25,10 +28,11 @@ namespace DiscordChatExporter.Core.Discord.Data
public static EmbedImage Parse(JsonElement json)
{
var url = json.GetPropertyOrNull("url")?.GetString();
var proxyUrl = json.GetPropertyOrNull("proxy_url")?.GetString();
var width = json.GetPropertyOrNull("width")?.GetInt32();
var height = json.GetPropertyOrNull("height")?.GetInt32();
return new EmbedImage(url, width, height);
return new EmbedImage(url, proxyUrl, width, height);
}
}
}
+26 -31
View File
@@ -1,7 +1,7 @@
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using Tyrrrz.Extensions;
@@ -10,21 +10,27 @@ namespace DiscordChatExporter.Core.Discord.Data
// https://discord.com/developers/docs/resources/emoji#emoji-object
public partial class Emoji
{
// Only present on custom emoji
public string? Id { get; }
// Name of custom emoji (e.g. LUL) or actual representation of standard emoji (e.g. 🙂)
public string Name { get; }
// Name of custom emoji (e.g. LUL) or name of standard emoji (e.g. slight_smile)
public string Code => !string.IsNullOrWhiteSpace(Id)
? Name
: EmojiIndex.TryGetCode(Name) ?? Name;
public bool IsAnimated { get; }
public string ImageUrl { get; }
public Emoji(string? id, string name, bool isAnimated)
public Emoji(string? id, string name, bool isAnimated, string imageUrl)
{
Id = id;
Name = name;
IsAnimated = isAnimated;
ImageUrl = GetImageUrl(id, name, isAnimated);
ImageUrl = imageUrl;
}
public override string ToString() => Name;
@@ -32,39 +38,26 @@ namespace DiscordChatExporter.Core.Discord.Data
public partial class Emoji
{
private static IEnumerable<Rune> GetRunes(string emoji)
{
var lastIndex = 0;
while (lastIndex < emoji.Length && Rune.TryGetRuneAt(emoji, lastIndex, out var rune))
{
// Skip variant selector rune
if (rune.Value != 0xfe0f)
yield return rune;
lastIndex += rune.Utf16SequenceLength;
}
}
private static string GetTwemojiName(IEnumerable<Rune> runes) =>
runes.Select(r => r.Value.ToString("x")).JoinToString("-");
private static string GetTwemojiName(string name) => name
.GetRunes()
// Variant selector rune is skipped in Twemoji names
.Where(r => r.Value != 0xfe0f)
.Select(r => r.Value.ToString("x"))
.JoinToString("-");
public static string GetImageUrl(string? id, string name, bool isAnimated)
{
// Custom emoji
if (!string.IsNullOrWhiteSpace(id))
{
// Animated
if (isAnimated)
return $"https://cdn.discordapp.com/emojis/{id}.gif";
// Non-animated
return $"https://cdn.discordapp.com/emojis/{id}.png";
return isAnimated
? $"https://cdn.discordapp.com/emojis/{id}.gif"
: $"https://cdn.discordapp.com/emojis/{id}.png";
}
// Standard emoji
var emojiRunes = GetRunes(name).ToArray();
var twemojiName = GetTwemojiName(emojiRunes);
return $"https://twemoji.maxcdn.com/2/72x72/{twemojiName}.png";
var twemojiName = GetTwemojiName(name);
return $"https://twemoji.maxcdn.com/2/svg/{twemojiName}.svg";
}
public static Emoji Parse(JsonElement json)
@@ -73,7 +66,9 @@ namespace DiscordChatExporter.Core.Discord.Data
var name = json.GetProperty("name").GetString();
var isAnimated = json.GetPropertyOrNull("animated")?.GetBoolean() ?? false;
return new Emoji(id, name, isAnimated);
var imageUrl = GetImageUrl(id, name, isAnimated);
return new Emoji(id, name, isAnimated, imageUrl);
}
}
}
@@ -19,10 +19,10 @@ namespace DiscordChatExporter.Core.Discord.Data
public IReadOnlyList<Snowflake> RoleIds { get; }
public Member(User user, string? nick, IReadOnlyList<Snowflake> roleIds)
public Member(User user, string nick, IReadOnlyList<Snowflake> roleIds)
{
User = user;
Nick = nick ?? user.Name;
Nick = nick;
RoleIds = roleIds;
}
@@ -33,7 +33,7 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public static Member CreateForUser(User user) => new(
user,
null,
user.Name,
Array.Empty<Snowflake>()
);
@@ -43,12 +43,12 @@ namespace DiscordChatExporter.Core.Discord.Data
var nick = json.GetPropertyOrNull("nick")?.GetString();
var roleIds =
json.GetPropertyOrNull("roles")?.EnumerateArray().Select(j => j.GetString().Pipe(Snowflake.Parse)).ToArray() ??
json.GetPropertyOrNull("roles")?.EnumerateArray().Select(j => j.GetString()).Select(Snowflake.Parse).ToArray() ??
Array.Empty<Snowflake>();
return new Member(
user,
nick,
nick ?? user.Name,
roleIds
);
}
@@ -100,7 +100,7 @@ namespace DiscordChatExporter.Core.Discord.Data
var type = (MessageType) json.GetProperty("type").GetInt32();
var isPinned = json.GetPropertyOrNull("pinned")?.GetBoolean() ?? false;
var messageReference = json.GetPropertyOrNull("message_reference")?.Pipe(MessageReference.Parse);
var referencedMessage = json.GetPropertyOrNull("referenced_message")?.Pipe(Message.Parse);
var referencedMessage = json.GetPropertyOrNull("referenced_message")?.Pipe(Parse);
var content = type switch
{
@@ -23,8 +23,8 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public static Reaction Parse(JsonElement json)
{
var count = json.GetProperty("count").GetInt32();
var emoji = json.GetProperty("emoji").Pipe(Emoji.Parse);
var count = json.GetProperty("count").GetInt32();
return new Reaction(emoji, count);
}
@@ -40,7 +40,8 @@ namespace DiscordChatExporter.Core.Discord.Data
var name = json.GetProperty("name").GetString();
var position = json.GetProperty("position").GetInt32();
var color = json.GetPropertyOrNull("color")?
var color = json
.GetPropertyOrNull("color")?
.GetInt32()
.Pipe(System.Drawing.Color.FromArgb)
.ResetAlpha()
@@ -49,19 +49,19 @@ namespace DiscordChatExporter.Core.Discord.Data
{
// Animated
if (avatarHash.StartsWith("a_", StringComparison.Ordinal))
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.gif";
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.gif?size=40";
// Non-animated
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.png";
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.png?size=40";
}
public static User Parse(JsonElement json)
{
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);
var isBot = json.GetPropertyOrNull("bot")?.GetBoolean() ?? false;
var discriminator = json.GetProperty("discriminator").GetString().Pipe(int.Parse);
var name = json.GetProperty("username").GetString();
var avatarHash = json.GetProperty("avatar").GetString();
var isBot = json.GetPropertyOrNull("bot")?.GetBoolean() ?? false;
var avatarUrl = !string.IsNullOrWhiteSpace(avatarHash)
? GetAvatarUrl(id, avatarHash)
@@ -0,0 +1,61 @@
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data
{
public partial class YouTubeVideoEmbedProjection
{
public string VideoId { get; }
public string Url { get; }
public YouTubeVideoEmbedProjection(string videoId, string url)
{
VideoId = videoId;
Url = url;
}
public override string ToString() => Url;
}
public partial class YouTubeVideoEmbedProjection
{
// Adapted from YoutubeExplode
// https://github.com/Tyrrrz/YoutubeExplode/blob/5be164be20019783913f76fcc98f18c65aebe9f0/YoutubeExplode/Videos/VideoId.cs#L34-L64
private static string? TryParseVideoId(string embedUrl)
{
// Regular URL
// https://www.youtube.com/watch?v=yIVRs6YSbOM
var regularMatch = Regex.Match(embedUrl, @"youtube\..+?/watch.*?v=(.*?)(?:&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(regularMatch))
return regularMatch;
// Short URL
// https://youtu.be/yIVRs6YSbOM
var shortMatch = Regex.Match(embedUrl, @"youtu\.be/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(shortMatch))
return shortMatch;
// Embed URL
// https://www.youtube.com/embed/yIVRs6YSbOM
var embedMatch = Regex.Match(embedUrl, @"youtube\..+?/embed/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(embedMatch))
return embedMatch;
return null;
}
public static YouTubeVideoEmbedProjection? TryResolve(Embed embed)
{
if (string.IsNullOrWhiteSpace(embed.Url))
return null;
var videoId = TryParseVideoId(embed.Url);
if (string.IsNullOrWhiteSpace(videoId))
return null;
var url = $"https://www.youtube.com/embed/{videoId}";
return new YouTubeVideoEmbedProjection(videoId, url);
}
}
}
@@ -11,7 +11,6 @@ using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Http;
using JsonExtensions.Reading;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Discord
{
@@ -29,7 +28,9 @@ namespace DiscordChatExporter.Core.Discord
}
public DiscordClient(AuthToken token)
: this(Http.Client, token) {}
: this(Http.Client, token)
{
}
private async ValueTask<HttpResponseMessage> GetResponseAsync(string url) =>
await Http.ResponsePolicy.ExecuteAsync(async () =>
@@ -64,7 +65,7 @@ namespace DiscordChatExporter.Core.Discord
return response.IsSuccessStatusCode
? await response.Content.ReadAsJsonAsync()
: (JsonElement?) null;
: null;
}
public async IAsyncEnumerable<Guild> GetUserGuildsAsync()
@@ -118,29 +119,30 @@ namespace DiscordChatExporter.Core.Discord
{
var response = await GetJsonResponseAsync($"guilds/{guildId}/channels");
var orderedResponse = response
var responseOrdered = response
.EnumerateArray()
.OrderBy(j => j.GetProperty("position").GetInt32())
.ThenBy(j => ulong.Parse(j.GetProperty("id").GetString()))
.ThenBy(j => Snowflake.Parse(j.GetProperty("id").GetString()))
.ToArray();
var categories = orderedResponse
.Where(j => j.GetProperty("type").GetInt32() == (int)ChannelType.GuildCategory)
var categories = responseOrdered
.Where(j => j.GetProperty("type").GetInt32() == (int) ChannelType.GuildCategory)
.Select((j, index) => ChannelCategory.Parse(j, index + 1))
.ToDictionary(j => j.Id.ToString());
.ToDictionary(j => j.Id.ToString(), StringComparer.Ordinal);
var position = 0;
foreach (var channelJson in orderedResponse)
foreach (var channelJson in responseOrdered)
{
var parentId = channelJson.GetPropertyOrNull("parent_id")?.GetString();
var category = !string.IsNullOrWhiteSpace(parentId)
? categories.GetValueOrDefault(parentId)
: null;
var channel = Channel.Parse(channelJson, category, position);
// Skip non-text channels
// We are only interested in text channels
if (!channel.IsTextChannel)
continue;
@@ -178,15 +180,12 @@ namespace DiscordChatExporter.Core.Discord
var response = await GetJsonResponseAsync($"channels/{channelId}");
return ChannelCategory.Parse(response);
}
/***
* In some cases, the Discord API returns an empty body when requesting some channel category info.
* Instead, we use an empty channel category as a fallback.
*/
// In some cases, the Discord API returns an empty body when requesting channel category.
// Instead, we use an empty channel category as a fallback.
catch (DiscordChatExporterException)
{
return ChannelCategory.Empty;
}
}
public async ValueTask<Channel> GetChannelAsync(Snowflake channelId)
@@ -221,7 +220,7 @@ namespace DiscordChatExporter.Core.Discord
IProgress<double>? progress = null)
{
// Get the last message in the specified range.
// This snapshots the boundaries, which means that messages posted after the exported started
// This snapshots the boundaries, which means that messages posted after the export started
// will not appear in the output.
// Additionally, it provides the date of the last message, which is used to calculate progress.
var lastMessage = await TryGetLastMessageAsync(channelId, before);
@@ -271,7 +270,7 @@ namespace DiscordChatExporter.Core.Discord
progress.Report(exportedDuration / totalDuration);
}
// Avoid division by zero if all messages have the exact same timestamp
// (which can happen easily if there's only one message in the channel)
// (which may be the case if there's only one message in the channel)
else
{
progress.Report(1);
@@ -284,4 +283,4 @@ namespace DiscordChatExporter.Core.Discord
}
}
}
}
}
@@ -10,8 +10,9 @@ namespace DiscordChatExporter.Core.Discord
public Snowflake(ulong value) => Value = value;
public DateTimeOffset ToDate() =>
DateTimeOffset.FromUnixTimeMilliseconds((long) ((Value >> 22) + 1420070400000UL)).ToLocalTime();
public DateTimeOffset ToDate() => DateTimeOffset.FromUnixTimeMilliseconds(
(long) ((Value >> 22) + 1420070400000UL)
).ToLocalTime();
public override string ToString() => Value.ToString(CultureInfo.InvariantCulture);
}
@@ -53,9 +54,11 @@ namespace DiscordChatExporter.Core.Discord
public static Snowflake Parse(string str) => Parse(str, null);
}
public partial struct Snowflake : IEquatable<Snowflake>
public partial struct Snowflake : IComparable<Snowflake>, IEquatable<Snowflake>
{
public bool Equals(Snowflake other) => Value == other.Value;
public int CompareTo(Snowflake other) => Value.CompareTo(other.Value);
public bool Equals(Snowflake other) => CompareTo(other) == 0;
public override bool Equals(object? obj) => obj is Snowflake other && Equals(other);
@@ -5,7 +5,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="JsonExtensions" Version="1.0.1" />
<PackageReference Include="JsonExtensions" Version="1.1.0" />
<PackageReference Include="MiniRazor.CodeGen" Version="2.1.2" />
<PackageReference Include="Polly" Version="7.2.2" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.5" />
@@ -79,7 +79,7 @@ namespace DiscordChatExporter.Core.Exporting
: filePath;
// HACK: for HTML, we need to format the URL properly
if (Request.Format == ExportFormat.HtmlDark || Request.Format == ExportFormat.HtmlLight)
if (Request.Format is ExportFormat.HtmlDark or ExportFormat.HtmlLight)
{
// Need to escape each path segment while keeping the directory separators intact
return relativeFilePath
@@ -93,7 +93,7 @@ namespace DiscordChatExporter.Core.Exporting
// Try to catch only exceptions related to failed HTTP requests
// https://github.com/Tyrrrz/DiscordChatExporter/issues/332
// https://github.com/Tyrrrz/DiscordChatExporter/issues/372
catch (Exception ex) when (ex is HttpRequestException || ex is OperationCanceledException)
catch (Exception ex) when (ex is HttpRequestException or OperationCanceledException)
{
// TODO: add logging so we can be more liberal with catching exceptions
// We don't want this to crash the exporting process in case of failure
@@ -45,7 +45,7 @@ namespace DiscordChatExporter.Core.Exporting
Directory.CreateDirectory(_workingDirPath);
// This catches IOExceptions which is dangerous as we're working also with files
// This retries on IOExceptions which is dangerous as we're also working with files
await Http.ExceptionPolicy.ExecuteAsync(async () =>
{
// Download the file
@@ -56,17 +56,27 @@ namespace DiscordChatExporter.Core.Exporting
}
// Try to set the file date according to the last-modified header
var lastModified = response.Content.Headers.TryGetValue("Last-Modified")?.Pipe(s =>
DateTimeOffset.TryParse(s, CultureInfo.InvariantCulture, DateTimeStyles.None, out var date)
? date
: (DateTimeOffset?) null
);
if (lastModified is not null)
try
{
File.SetCreationTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastWriteTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastAccessTimeUtc(filePath, lastModified.Value.UtcDateTime);
var lastModified = response.Content.Headers.TryGetValue("Last-Modified")?.Pipe(s =>
DateTimeOffset.TryParse(s, CultureInfo.InvariantCulture, DateTimeStyles.None, out var date)
? date
: (DateTimeOffset?) null
);
if (lastModified is not null)
{
File.SetCreationTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastWriteTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastAccessTimeUtc(filePath, lastModified.Value.UtcDateTime);
}
}
catch
{
// This can apparently fail for some reason.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/585
// Updating file dates is not a critical task, so we'll just
// ignore exceptions thrown here.
}
});
@@ -102,4 +112,4 @@ namespace DiscordChatExporter.Core.Exporting
return PathEx.EscapePath(fileNameWithoutExtension.Truncate(42) + '-' + urlHash + fileExtension);
}
}
}
}
@@ -146,8 +146,8 @@ img {
}
.emoji {
width: 1.25em;
height: 1.25em;
width: 1.325em;
height: 1.325em;
margin: 0 0.06em;
vertical-align: -0.4em;
}
@@ -244,14 +244,21 @@ img {
flex-grow: 1;
overflow: hidden;
text-overflow: ellipsis;
}
.chatlog__reference-link:hover {
text-decoration: none;
cursor: pointer;
}
.chatlog__reference-content > * {
display: inline;
pointer-events: none;
}
.chatlog__reference-content > a:hover {
text-decoration: none;
}
.chatlog__reference-content > .emoji {
width: 1.325em;
height: 1.325em;
}
.chatlog__reference-edited-timestamp {
@@ -480,10 +487,20 @@ img {
}
.chatlog__embed-footer-text {
vertical-align: middle;
font-size: 0.75em;
font-weight: 500;
}
.chatlog__embed-youtube-container {
margin-top: 0.6em;
}
.chatlog__embed-youtube {
border: 0;
border-radius: 3px;
}
.chatlog__reactions {
display: flex;
}
@@ -510,7 +527,7 @@ img {
border-radius: 3px;
vertical-align: middle;
line-height: 1.3;
background: #7289da;
background: #5865F2;
color: #ffffff;
font-size: 0.625em;
font-weight: 500;
@@ -6,7 +6,7 @@ body {
}
a {
color: #0096cf;
color: #00aff4;
}
.spoiler-text {
@@ -34,7 +34,7 @@ a {
color: #b9bbbe !important;
}
/* === Preamble === */
/* Preamble */
.preamble__entry {
color: #ffffff;
@@ -92,7 +92,7 @@ a {
}
.chatlog__embed-color-pill--default {
background-color: rgba(79, 84, 92, 1);
background-color: #202225;
}
.chatlog__embed-content-container {
@@ -113,7 +113,7 @@ a {
}
.chatlog__embed-description {
color: rgba(255, 255, 255, 0.6);
color: #dcddde;
}
.chatlog__embed-field-name {
@@ -121,11 +121,11 @@ a {
}
.chatlog__embed-field-value {
color: rgba(255, 255, 255, 0.6);
color: #dcddde;
}
.chatlog__embed-footer {
color: rgba(255, 255, 255, 0.6);
color: #dcddde;
}
.chatlog__reaction {
@@ -7,7 +7,7 @@ body {
}
a {
color: #00b0f4;
color: #0068e0;
}
.spoiler-text {
@@ -115,7 +115,7 @@ a {
}
.chatlog__embed-description {
color: #737f8d;
color: #2e3338;
}
.chatlog__embed-field-name {
@@ -123,11 +123,11 @@ a {
}
.chatlog__embed-field-value {
color: #737f8d;
color: #2e3338;
}
.chatlog__embed-footer {
color: rgba(79, 83, 91, 0.6);
color: #2e3338;
}
.chatlog__reaction {
@@ -36,9 +36,13 @@ namespace DiscordChatExporter.Core.Exporting.Writers.Html
internal partial class MessageGroup
{
public static bool CanJoin(Message message1, Message message2) =>
// Must be from the same author
message1.Author.Id == message2.Author.Id &&
// Author's name must not have changed between messages
string.Equals(message1.Author.FullName, message2.Author.FullName, StringComparison.Ordinal) &&
// Duration between messages must be 7 minutes or less
(message2.Timestamp - message1.Timestamp).Duration().TotalMinutes <= 7 &&
// Other message must not be a reply
message2.Reference is null;
public static MessageGroup Join(IReadOnlyList<Message> messages)
@@ -54,7 +54,7 @@
{
<img class="chatlog__reference-avatar" src="@await ResolveUrlAsync(Model.MessageGroup.ReferencedMessage.Author.AvatarUrl)" alt="Avatar">
<span class="chatlog__reference-name" title="@Model.MessageGroup.ReferencedMessage.Author.FullName" style="@referencedUserColorStyle">@referencedUserNick</span>
<a class="chatlog__reference-link" href="#" onclick="scrollToMessage(event, '@Model.MessageGroup.ReferencedMessage.Id')">
<div class="chatlog__reference-link" onclick="scrollToMessage(event, '@Model.MessageGroup.ReferencedMessage.Id')">
<span class="chatlog__reference-content">
@if (!string.IsNullOrWhiteSpace(Model.MessageGroup.ReferencedMessage.Content))
{
@@ -70,7 +70,7 @@
{
<span class="chatlog__reference-edited-timestamp" title="@FormatDate(Model.MessageGroup.ReferencedMessage.EditedTimestamp.Value)">(edited)</span>
}
</a>
</div>
}
else
{
@@ -97,7 +97,7 @@
{
var isPinnedStyle = message.IsPinned ? "chatlog__message--pinned" : null;
<div class="chatlog__message @isPinnedStyle" data-message-id="@message.Id" id="message-@message.Id">
<div class="chatlog__message @isPinnedStyle" data-message-id="@message.Id" id="message-@message.Id" title="Message sent: @FormatDate(message.Timestamp)">
@if (!string.IsNullOrWhiteSpace(message.Content) || message.EditedTimestamp is not null)
{
<div class="chatlog__content">
@@ -161,6 +161,8 @@
@foreach (var embed in message.Embeds)
{
var youTubeVideo = embed.TryGetYouTubeVideo();
<div class="chatlog__embed">
@if (embed.Color is not null)
{
@@ -180,7 +182,7 @@
<div class="chatlog__embed-author">
@if (!string.IsNullOrWhiteSpace(embed.Author.IconUrl))
{
<img class="chatlog__embed-author-icon" src="@await ResolveUrlAsync(embed.Author.IconUrl)" alt="Author icon">
<img class="chatlog__embed-author-icon" src="@await ResolveUrlAsync(embed.Author.IconProxyUrl ?? embed.Author.IconUrl)" alt="Author icon">
}
@if (!string.IsNullOrWhiteSpace(embed.Author.Name))
@@ -215,7 +217,13 @@
</div>
}
@if (!string.IsNullOrWhiteSpace(embed.Description))
@if (youTubeVideo is not null)
{
<div class="chatlog__embed-youtube-container">
<iframe class="chatlog__embed-youtube" src="@youTubeVideo.Url" width="400" height="225"></iframe>
</div>
}
else if (!string.IsNullOrWhiteSpace(embed.Description))
{
<div class="chatlog__embed-description">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Description))</div>
@@ -249,21 +257,21 @@
}
</div>
@if (embed.Thumbnail is not null)
@if (embed.Thumbnail is not null && !string.IsNullOrWhiteSpace(embed.Thumbnail.Url) && youTubeVideo is null)
{
<div class="chatlog__embed-thumbnail-container">
<a class="chatlog__embed-thumbnail-link" href="@await ResolveUrlAsync(embed.Thumbnail.Url)">
<img class="chatlog__embed-thumbnail" src="@await ResolveUrlAsync(embed.Thumbnail.Url)" alt="Thumbnail">
<a class="chatlog__embed-thumbnail-link" href="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)">
<img class="chatlog__embed-thumbnail" src="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)" alt="Thumbnail">
</a>
</div>
}
</div>
@if (embed.Image is not null)
@if (embed.Image is not null && !string.IsNullOrWhiteSpace(embed.Image.Url))
{
<div class="chatlog__embed-image-container">
<a class="chatlog__embed-image-link" href="@await ResolveUrlAsync(embed.Image.Url)">
<img class="chatlog__embed-image" src="@await ResolveUrlAsync(embed.Image.Url)" alt="Image">
<a class="chatlog__embed-image-link" href="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)">
<img class="chatlog__embed-image" src="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)" alt="Image">
</a>
</div>
}
@@ -273,7 +281,7 @@
<div class="chatlog__embed-footer">
@if (!string.IsNullOrWhiteSpace(embed.Footer?.IconUrl))
{
<img class="chatlog__embed-footer-icon" src="@await ResolveUrlAsync(embed.Footer.IconUrl)" alt="Footer icon">
<img class="chatlog__embed-footer-icon" src="@await ResolveUrlAsync(embed.Footer.IconProxyUrl ?? embed.Footer.IconUrl)" alt="Footer icon">
}
<span class="chatlog__embed-footer-text">
@@ -303,8 +311,8 @@
<div class="chatlog__reactions">
@foreach (var reaction in message.Reactions)
{
<div class="chatlog__reaction">
<img class="emoji emoji--small" alt="@reaction.Emoji.Name" title="@reaction.Emoji.Name" src="@await ResolveUrlAsync(reaction.Emoji.ImageUrl)">
<div class="chatlog__reaction" title="@reaction.Emoji.Code">
<img class="emoji emoji--small" alt="@reaction.Emoji.Name" src="@await ResolveUrlAsync(reaction.Emoji.ImageUrl)">
<span class="chatlog__reaction-count">@reaction.Count</span>
</div>
}
@@ -20,6 +20,8 @@ namespace DiscordChatExporter.Core.Exporting.Writers
{
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
Indented = true,
// Validation errors may mask actual failures
// https://github.com/Tyrrrz/DiscordChatExporter/issues/413
SkipValidation = true
});
}
@@ -48,7 +50,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteString("url", embedAuthor.Url);
if (!string.IsNullOrWhiteSpace(embedAuthor.IconUrl))
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedAuthor.IconUrl));
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedAuthor.IconProxyUrl ?? embedAuthor.IconUrl));
_writer.WriteEndObject();
await _writer.FlushAsync();
@@ -59,7 +61,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteStartObject("thumbnail");
if (!string.IsNullOrWhiteSpace(embedThumbnail.Url))
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedThumbnail.Url));
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedThumbnail.ProxyUrl ?? embedThumbnail.Url));
_writer.WriteNumber("width", embedThumbnail.Width);
_writer.WriteNumber("height", embedThumbnail.Height);
@@ -73,7 +75,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteStartObject("image");
if (!string.IsNullOrWhiteSpace(embedImage.Url))
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedImage.Url));
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedImage.ProxyUrl ?? embedImage.Url));
_writer.WriteNumber("width", embedImage.Width);
_writer.WriteNumber("height", embedImage.Height);
@@ -89,7 +91,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteString("text", embedFooter.Text);
if (!string.IsNullOrWhiteSpace(embedFooter.IconUrl))
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedFooter.IconUrl));
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedFooter.IconProxyUrl ?? embedFooter.IconUrl));
_writer.WriteEndObject();
await _writer.FlushAsync();
@@ -300,4 +302,4 @@ namespace DiscordChatExporter.Core.Exporting.Writers
await base.DisposeAsync();
}
}
}
}
@@ -131,7 +131,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
var jumboClass = _isJumbo ? "emoji--large" : "";
_buffer
.Append($"<img class=\"emoji {jumboClass}\" alt=\"{emoji.Name}\" title=\"{emoji.Name}\" src=\"{emojiImageUrl}\">");
.Append($"<img class=\"emoji {jumboClass}\" alt=\"{emoji.Name}\" title=\"{emoji.Code}\" src=\"{emojiImageUrl}\">");
return base.VisitEmoji(emoji);
}
@@ -75,10 +75,10 @@ namespace DiscordChatExporter.Core.Exporting.Writers
}
if (!string.IsNullOrWhiteSpace(embed.Thumbnail?.Url))
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Thumbnail.Url));
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url));
if (!string.IsNullOrWhiteSpace(embed.Image?.Url))
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Image.Url));
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url));
if (!string.IsNullOrWhiteSpace(embed.Footer?.Text))
await _writer.WriteLineAsync(embed.Footer.Text);
@@ -1,11 +1,20 @@
namespace DiscordChatExporter.Core.Markdown.Ast
using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Markdown.Ast
{
internal class EmojiNode : MarkdownNode
{
// Only present on custom emoji
public string? Id { get; }
// Name of custom emoji (e.g. LUL) or actual representation of standard emoji (e.g. 🙂)
public string Name { get; }
// Name of custom emoji (e.g. LUL) or name of standard emoji (e.g. slight_smile)
public string Code => !string.IsNullOrWhiteSpace(Id)
? Name
: EmojiIndex.TryGetCode(Name) ?? Name;
public bool IsAnimated { get; }
public bool IsCustomEmoji => !string.IsNullOrWhiteSpace(Id);
@@ -2,16 +2,6 @@
namespace DiscordChatExporter.Core.Markdown.Ast
{
internal enum TextFormatting
{
Bold,
Italic,
Underline,
Strikethrough,
Spoiler,
Quote
}
internal class FormattedNode : MarkdownNode
{
public TextFormatting Formatting { get; }
@@ -1,13 +1,5 @@
namespace DiscordChatExporter.Core.Markdown.Ast
{
internal enum MentionType
{
Meta,
User,
Channel,
Role
}
internal class MentionNode : MarkdownNode
{
public string Id { get; }
@@ -0,0 +1,10 @@
namespace DiscordChatExporter.Core.Markdown.Ast
{
internal enum MentionType
{
Meta,
User,
Channel,
Role
}
}
@@ -0,0 +1,12 @@
namespace DiscordChatExporter.Core.Markdown.Ast
{
internal enum TextFormatting
{
Bold,
Italic,
Underline,
Strikethrough,
Spoiler,
Quote
}
}
@@ -3,6 +3,7 @@ using System.Linq;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Markdown.Ast;
using DiscordChatExporter.Core.Markdown.Matching;
using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Markdown
{
@@ -103,7 +104,7 @@ namespace DiscordChatExporter.Core.Markdown
// There can be either one or two backticks, but equal number on both sides
private static readonly IMatcher<MarkdownNode> InlineCodeBlockNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(`{1,2})([^`]+)\\1", DefaultRegexOptions | RegexOptions.Singleline),
m => new InlineCodeBlockNode(m.Groups[2].Value.Trim('\r', '\n'))
(_, m) => new InlineCodeBlockNode(m.Groups[2].Value.Trim('\r', '\n'))
);
// Capture language identifier and then any character until the earliest triple backtick
@@ -111,7 +112,7 @@ namespace DiscordChatExporter.Core.Markdown
// Blank lines at the beginning and end of content are trimmed
private static readonly IMatcher<MarkdownNode> MultiLineCodeBlockNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("```(?:(\\w*)\\n)?(.+?)```", DefaultRegexOptions | RegexOptions.Singleline),
m => new MultiLineCodeBlockNode(m.Groups[1].Value, m.Groups[2].Value.Trim('\r', '\n'))
(_, m) => new MultiLineCodeBlockNode(m.Groups[1].Value, m.Groups[2].Value.Trim('\r', '\n'))
);
/* Mentions */
@@ -131,19 +132,19 @@ namespace DiscordChatExporter.Core.Markdown
// Capture <@123456> or <@!123456>
private static readonly IMatcher<MarkdownNode> UserMentionNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<@!?(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Groups[1].Value, MentionType.User)
(_, m) => new MentionNode(m.Groups[1].Value, MentionType.User)
);
// Capture <#123456>
private static readonly IMatcher<MarkdownNode> ChannelMentionNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<#(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Groups[1].Value, MentionType.Channel)
(_, m) => new MentionNode(m.Groups[1].Value, MentionType.Channel)
);
// Capture <@&123456>
private static readonly IMatcher<MarkdownNode> RoleMentionNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<@&(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Groups[1].Value, MentionType.Role)
(_, m) => new MentionNode(m.Groups[1].Value, MentionType.Role)
);
/* Emojis */
@@ -154,15 +155,26 @@ namespace DiscordChatExporter.Core.Markdown
// ... or digit followed by enclosing mark
// (this does not match all emojis in Discord but it's reasonably accurate enough)
private static readonly IMatcher<MarkdownNode> StandardEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("((?:[\\uD83C][\\uDDE6-\\uDDFF]){2}|[\\u2600-\\u26FF]|\\p{Cs}{2}|\\d\\p{Me})",
DefaultRegexOptions),
m => new EmojiNode(m.Groups[1].Value)
new Regex("((?:[\\uD83C][\\uDDE6-\\uDDFF]){2}|[\\u2600-\\u26FF]|\\p{Cs}{2}|\\d\\p{Me})", DefaultRegexOptions),
(_, m) => new EmojiNode(m.Groups[1].Value)
);
// Capture :thinking: (but only for known emoji codes)
private static readonly IMatcher<MarkdownNode> CodedStandardEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex(":([\\w_]+):", DefaultRegexOptions),
(_, m) =>
{
var name = EmojiIndex.TryGetName(m.Groups[1].Value);
return name is not null
? new EmojiNode(name)
: null;
}
);
// Capture <:lul:123456> or <a:lul:123456>
private static readonly IMatcher<MarkdownNode> CustomEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<(a)?:(.+?):(\\d+?)>", DefaultRegexOptions),
m => new EmojiNode(m.Groups[3].Value, m.Groups[2].Value, !string.IsNullOrWhiteSpace(m.Groups[1].Value))
(_, m) => new EmojiNode(m.Groups[3].Value, m.Groups[2].Value, !string.IsNullOrWhiteSpace(m.Groups[1].Value))
);
/* Links */
@@ -170,19 +182,19 @@ namespace DiscordChatExporter.Core.Markdown
// Capture [title](link)
private static readonly IMatcher<MarkdownNode> TitledLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\[(.+?)\\]\\((.+?)\\)", DefaultRegexOptions),
m => new LinkNode(m.Groups[2].Value, m.Groups[1].Value)
(_, m) => new LinkNode(m.Groups[2].Value, m.Groups[1].Value)
);
// Capture any non-whitespace character after http:// or https:// until the last punctuation character or whitespace
private static readonly IMatcher<MarkdownNode> AutoLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(https?://\\S*[^\\.,:;\"\'\\s])", DefaultRegexOptions),
m => new LinkNode(m.Groups[1].Value)
(_, m) => new LinkNode(m.Groups[1].Value)
);
// Same as auto link but also surrounded by angular brackets
private static readonly IMatcher<MarkdownNode> HiddenLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<(https?://\\S*[^\\.,:;\"\'\\s])>", DefaultRegexOptions),
m => new LinkNode(m.Groups[1].Value)
(_, m) => new LinkNode(m.Groups[1].Value)
);
/* Text */
@@ -198,21 +210,21 @@ namespace DiscordChatExporter.Core.Markdown
// This escapes it from matching for emoji
private static readonly IMatcher<MarkdownNode> IgnoredEmojiTextNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(\\u26A7|\\u2640|\\u2642|\\u2695|\\u267E|\\u00A9|\\u00AE|\\u2122)", DefaultRegexOptions),
m => new TextNode(m.Groups[1].Value)
(_, m) => new TextNode(m.Groups[1].Value)
);
// Capture any "symbol/other" character or surrogate pair preceded by a backslash
// This escapes it from matching for emoji
private static readonly IMatcher<MarkdownNode> EscapedSymbolTextNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\\\(\\p{So}|\\p{Cs}{2})", DefaultRegexOptions),
m => new TextNode(m.Groups[1].Value)
(_, m) => new TextNode(m.Groups[1].Value)
);
// Capture any non-whitespace, non latin alphanumeric character preceded by a backslash
// This escapes it from matching for formatting or other tokens
private static readonly IMatcher<MarkdownNode> EscapedCharacterTextNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\\\([^a-zA-Z0-9\\s])", DefaultRegexOptions),
m => new TextNode(m.Groups[1].Value)
(_, m) => new TextNode(m.Groups[1].Value)
);
// Combine all matchers into one
@@ -255,7 +267,8 @@ namespace DiscordChatExporter.Core.Markdown
// Emoji
StandardEmojiNodeMatcher,
CustomEmojiNodeMatcher
CustomEmojiNodeMatcher,
CodedStandardEmojiNodeMatcher
);
// Minimal set of matchers for non-multimedia formats (e.g. plain text)
@@ -6,19 +6,14 @@ namespace DiscordChatExporter.Core.Markdown.Matching
internal class RegexMatcher<T> : IMatcher<T>
{
private readonly Regex _regex;
private readonly Func<StringPart, Match, T> _transform;
private readonly Func<StringPart, Match, T?> _transform;
public RegexMatcher(Regex regex, Func<StringPart, Match, T> transform)
public RegexMatcher(Regex regex, Func<StringPart, Match, T?> transform)
{
_regex = regex;
_transform = transform;
}
public RegexMatcher(Regex regex, Func<Match, T> transform)
: this(regex, (p, m) => transform(m))
{
}
public ParsedMatch<T>? TryMatch(StringPart stringPart)
{
var match = _regex.Match(stringPart.Target, stringPart.StartIndex, stringPart.Length);
@@ -30,11 +25,15 @@ namespace DiscordChatExporter.Core.Markdown.Matching
// Which is super weird because regex.Match(string, int) takes the whole input in context.
// So in order to properly account for ^/$ regex tokens, we need to make sure that
// the expression also matches on the bigger part of the input.
if (!_regex.IsMatch(stringPart.Target.Substring(0, stringPart.EndIndex), stringPart.StartIndex))
if (!_regex.IsMatch(stringPart.Target[..stringPart.EndIndex], stringPart.StartIndex))
return null;
var stringPartMatch = stringPart.Slice(match.Index, match.Length);
return new ParsedMatch<T>(stringPartMatch, _transform(stringPartMatch, match));
var value = _transform(stringPartMatch, match);
return value is not null
? new ParsedMatch<T>(stringPartMatch, value)
: null;
}
}
}
@@ -6,9 +6,9 @@ namespace DiscordChatExporter.Core.Markdown.Matching
{
private readonly string _needle;
private readonly StringComparison _comparison;
private readonly Func<StringPart, T> _transform;
private readonly Func<StringPart, T?> _transform;
public StringMatcher(string needle, StringComparison comparison, Func<StringPart, T> transform)
public StringMatcher(string needle, StringComparison comparison, Func<StringPart, T?> transform)
{
_needle = needle;
_comparison = comparison;
@@ -23,14 +23,15 @@ namespace DiscordChatExporter.Core.Markdown.Matching
public ParsedMatch<T>? TryMatch(StringPart stringPart)
{
var index = stringPart.Target.IndexOf(_needle, stringPart.StartIndex, stringPart.Length, _comparison);
if (index < 0)
return null;
if (index >= 0)
{
var stringPartMatch = stringPart.Slice(index, _needle.Length);
return new ParsedMatch<T>(stringPartMatch, _transform(stringPartMatch));
}
var stringPartMatch = stringPart.Slice(index, _needle.Length);
var value = _transform(stringPartMatch);
return null;
return value is not null
? new ParsedMatch<T>(stringPartMatch, value)
: null;
}
}
}
@@ -10,14 +10,13 @@ namespace DiscordChatExporter.Core.Markdown.Matching
public int Length { get; }
public int EndIndex { get; }
public int EndIndex => StartIndex + Length;
public StringPart(string target, int startIndex, int length)
{
Target = target;
StartIndex = startIndex;
Length = length;
EndIndex = startIndex + length;
}
public StringPart(string target)
File diff suppressed because it is too large Load Diff
@@ -9,6 +9,6 @@ namespace DiscordChatExporter.Core.Utils.Extensions
public static T? NullIf<T>(this T value, Func<T, bool> predicate) where T : struct =>
!predicate(value)
? value
: (T?) null;
: null;
}
}
@@ -1,14 +1,30 @@
using System.Text;
using System.Collections.Generic;
using System.Text;
namespace DiscordChatExporter.Core.Utils.Extensions
{
public static class StringExtensions
{
public static string? NullIfWhiteSpace(this string str) =>
!string.IsNullOrWhiteSpace(str)
? str
: null;
public static string Truncate(this string str, int charCount) =>
str.Length > charCount
? str.Substring(0, charCount)
? str[..charCount]
: str;
public static IEnumerable<Rune> GetRunes(this string str)
{
var lastIndex = 0;
while (lastIndex < str.Length && Rune.TryGetRuneAt(str, lastIndex, out var rune))
{
yield return rune;
lastIndex += rune.Utf16SequenceLength;
}
}
public static StringBuilder AppendIfNotEmpty(this StringBuilder builder, char value) =>
builder.Length > 0
? builder.Append(value)
+14 -10
View File
@@ -5,6 +5,7 @@ using System.Net;
using System.Net.Http;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Utils.Extensions;
using Polly;
namespace DiscordChatExporter.Core.Utils
@@ -41,21 +42,24 @@ namespace DiscordChatExporter.Core.Utils
},
(_, _, _, _) => Task.CompletedTask);
private static HttpStatusCode? TryGetStatusCodeFromException(HttpRequestException ex)
{
private static HttpStatusCode? TryGetStatusCodeFromException(HttpRequestException ex) =>
// This is extremely frail, but there's no other way
var statusCodeRaw = Regex.Match(ex.Message, @": (\d+) \(").Groups[1].Value;
return !string.IsNullOrWhiteSpace(statusCodeRaw)
? (HttpStatusCode) int.Parse(statusCodeRaw, CultureInfo.InvariantCulture)
: (HttpStatusCode?) null;
}
Regex
.Match(ex.Message, @": (\d+) \(")
.Groups[1]
.Value
.NullIfWhiteSpace()?
.Pipe(s => (HttpStatusCode) int.Parse(s, CultureInfo.InvariantCulture));
public static IAsyncPolicy ExceptionPolicy { get; } =
Policy
.Handle<IOException>() // dangerous
.Or<HttpRequestException>(ex => TryGetStatusCodeFromException(ex) == HttpStatusCode.TooManyRequests)
.Or<HttpRequestException>(ex => TryGetStatusCodeFromException(ex) == HttpStatusCode.RequestTimeout)
.Or<HttpRequestException>(ex => TryGetStatusCodeFromException(ex) >= HttpStatusCode.InternalServerError)
.Or<HttpRequestException>(ex =>
TryGetStatusCodeFromException(ex) is
HttpStatusCode.TooManyRequests or
HttpStatusCode.RequestTimeout or
HttpStatusCode.InternalServerError
)
.WaitAndRetryAsync(4, i => TimeSpan.FromSeconds(Math.Pow(2, i) + 1));
}
}
@@ -11,9 +11,13 @@
<Resource Include="../favicon.ico" />
</ItemGroup>
<ItemGroup Condition="'$(Configuration)' == 'Release'">
<PackageReference Include="DotnetRuntimeBootstrapper" Version="1.1.2" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Gress" Version="1.2.0" />
<PackageReference Include="MaterialDesignColors" Version="2.0.0" />
<PackageReference Include="MaterialDesignColors" Version="2.0.1" />
<PackageReference Include="MaterialDesignThemes" Version="4.0.0" />
<PackageReference Include="Microsoft.Xaml.Behaviors.Wpf" Version="1.1.31" />
<PackageReference Include="Ookii.Dialogs.Wpf" Version="3.1.0" />
@@ -21,7 +25,7 @@
<PackageReference Include="Stylet" Version="1.3.6" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.5" />
<PackageReference Include="Tyrrrz.Settings" Version="1.3.4" />
<PackageReference Include="PropertyChanged.Fody" Version="3.3.3" PrivateAssets="all" />
<PackageReference Include="PropertyChanged.Fody" Version="3.4.0" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
+11 -27
View File
@@ -23,7 +23,6 @@ The following table lists all available download options:
<th></th>
<th>Downloads</th>
<th>Supported OS</th>
<th>Requirements</th>
</tr>
</thead>
<tbody>
@@ -40,49 +39,34 @@ The following table lists all available download options:
<li>Windows <b>7</b> or higher</li>
</ul>
</td>
<td>
<div>.NET v3.1 Desktop Runtime</div>
<ul>
<li><a href="https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-desktop-3.1.0-windows-x64-installer">Windows <b>x64</b></a></li>
<li><a href="https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-desktop-3.1.0-windows-x86-installer">Windows <b>x86</b></a></li>
</ul>
</td>
</tr>
<tr>
<td><b>CLI</b></td>
<td>
<ul>
<li>🟢 <b><a href="https://github.com/Tyrrrz/DiscordChatExporter/releases/latest">Stable release</a></b> (<code>DiscordChatExporter.CLI.zip</code>)</li>
<li>🟠 <a href="https://github.com/Tyrrrz/DiscordChatExporter/actions?query=workflow%3ACI">CI build</a> (<code>DiscordChatExporter.CLI.zip</code>)</li>
<li>🐋 <a href="https://hub.docker.com/r/tyrrrz/discordchatexporter">Docker</a> (<code>tyrrrz/discordchatexporter</code>) 🚀</li>
<li>📦 <a href="https://aur.archlinux.org/packages/discord-chat-exporter-cli">AUR</a> (<code>discord-chat-exporter-cli</code>) 🚀 🦄</li>
<li>🟢 <b><a href="https://github.com/Tyrrrz/DiscordChatExporter/releases/latest">Stable release</a></b> (<code>DiscordChatExporter.CLI.zip</code>) ⚙️</li>
<li>🟠 <a href="https://github.com/Tyrrrz/DiscordChatExporter/actions?query=workflow%3ACI">CI build</a> (<code>DiscordChatExporter.CLI.zip</code>) ⚙️</li>
<li>🐋 <a href="https://hub.docker.com/r/tyrrrz/discordchatexporter">Docker</a> (<code>tyrrrz/discordchatexporter</code>)</li>
<li>📦 <a href="https://aur.archlinux.org/packages/discord-chat-exporter-cli">AUR</a> (<code>discord-chat-exporter-cli</code>) 🦄</li>
</ul>
</td>
<td>
<ul>
<li>Windows <b>7</b> or higher</li>
<li>macOS <b>10.13 (High Sierra)</b> or higher</li>
<li>Linux</li>
</ul>
</td>
<td>
<div>.NET v3.1 Base Runtime</div>
<ul>
<li><a href="https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-desktop-3.1.0-windows-x64-installer">Windows <b>x64</b></a></li>
<li><a href="https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-desktop-3.1.0-windows-x86-installer">Windows <b>x86</b></a></li>
<li><a href="https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-3.1.0-macos-x64-installer">macOS <b>x64</b></a></li>
<li><a href="https://docs.microsoft.com/en-us/dotnet/core/install/linux">Linux (find your distribution)</a></li>
<li>Linux (multiple distros)</li>
</ul>
</td>
</tr>
</tbody>
</table>
- 🚀 - .NET Runtime is already embedded or installed automatically
- 🦄 - Community-maintained resource
> Note that DiscordChatExporter requires **.NET v3.1 Runtime** in order to work!
Refer to the **Requirements** column in the above table to download the appropriate installer for your system.
- ⚙️ - Requires .NET runtime to be installed manually:
- [.NET Core v3.1 Runtime for **Windows x64**](https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-desktop-3.1.0-windows-x64-installer)
- [.NET Core v3.1 Runtime for **Windows x86**](https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-desktop-3.1.0-windows-x86-installer)
- [.NET Core v3.1 Runtime for **macOS x64**](https://dotnet.microsoft.com/download/dotnet-core/thank-you/runtime-3.1.0-macos-x64-installer)
- [.NET Core v3.1 Runtime for **Linux**](https://docs.microsoft.com/en-us/dotnet/core/install/linux) (find your distro)
## Features
@@ -98,4 +82,4 @@ Refer to the **Requirements** column in the above table to download the appropri
## Screenshots
![channel list](.screenshots/list.png)
![rendered output](.screenshots/output.png)
![rendered output](.screenshots/output.png)