Compare commits

...

30 Commits

Author SHA1 Message Date
Alexey Golub 9c043376f9 Update version 2019-04-12 00:14:54 +03:00
Alexey Golub 77366cc9b4 Escape specific emojis in markdown
Fixes a part of #146
2019-04-12 00:11:01 +03:00
Alexey Golub ffdca7b799 [HTML] Fix multiple emojis not getting jumbo 2019-04-11 23:31:29 +03:00
Alexey Golub 30cba7959f Refactor string checks and fix exception when exporting multiple channels
Fix #164
2019-04-11 22:56:29 +03:00
Alexey Golub 7951703cf2 Update version 2019-04-11 01:52:50 +03:00
Alexey Golub 795dfbb213 [CLI] Don't show 'completed' when export failed 2019-04-11 01:36:39 +03:00
Alexey Golub 6d9bc3625f Switch from DateTime to DateTimeOffset 2019-04-11 01:20:52 +03:00
Alexey Golub 4bfb2ec7fd Improve performance (#162) 2019-04-10 23:45:21 +03:00
Oleksii Holub 359278afec Optimize chocolatey package 2019-04-09 18:15:17 +03:00
Oleksii Holub 19f58f2c8c Update readme 2019-04-09 16:32:24 +03:00
Alexey Golub 640b1f5866 Add deploy via chocolatey 2019-04-07 13:01:38 +03:00
Oleksii Holub d13d04e75b Add donation info 2019-04-03 18:33:34 +03:00
Oleksii Holub 6bf0ce3790 Fix dockerfile again 2019-03-25 15:22:51 +02:00
Oleksii Holub aef8a1f6d2 Fix default output directory in dockerfile 2019-03-25 14:35:24 +02:00
Oleksii Holub a63bf95212 Update dockerfile 2019-03-25 11:04:51 +02:00
Oleksii Holub af9855fc15 [CLI] Copy nuget dependencies to output 2019-03-25 11:00:13 +02:00
Alexey Golub 4113101c0c Get rid of ByteSize in favor of our own class 2019-03-24 12:36:25 +02:00
Oleksii Holub cac48b3541 Optimize dockerfile 2019-03-22 15:09:07 +02:00
Alexey Golub 0a60dea5c1 Use net46, netstd2.0, and netcoreapp2.1 for target frameworks
Closes #154
2019-03-22 00:07:13 +02:00
Alexey Golub 889446fff7 Update nuget packages 2019-03-21 22:54:12 +02:00
Alexey Golub 6d1cfef729 Update Onova 2019-03-21 00:55:24 +02:00
Oleksii Holub 77bf32aa5a Update version 2019-03-14 16:36:55 +02:00
Oleksii Holub ac2b450892 Retry on any 5xx HTTP response in DataService
Closes #157
2019-03-14 16:30:27 +02:00
Oleksii Holub 4046fd0ad9 [HTML] Add syntax highlighting via Highlight.js
Closes #138
2019-03-14 16:02:39 +02:00
Alexey Golub 6fa61bf75a Update version 2019-03-10 14:45:39 +02:00
Alexey Golub 1976e3ad08 [CLI] Don't write progress if output is redirected
FIxes #155
2019-03-10 14:40:08 +02:00
Alexey Golub adbe544c57 Add some margin at the end of the chatlog in HTML 2019-03-07 22:15:21 +02:00
Alexey Golub 6f4d1c3d77 Refactor message grouping 2019-03-07 22:13:00 +02:00
Alexey Golub 33f71c1612 Remove message group limit setting 2019-03-07 21:14:11 +02:00
Alexey Golub 9b78472e05 Update message grouping algorithm to match Discord's
Fixes #152
2019-03-07 21:10:22 +02:00
101 changed files with 1720 additions and 1086 deletions
+23
View File
@@ -1,3 +1,26 @@
### v2.12.1 (12-Apr-2019)
- [GUI] Fixed an issue where the app crashed when trying to export multiple channels.
- [HTML] Fixed an issue where some of the emojis were rendered via Twemoji while Discord renders them as plain text.
### v2.12 (11-Apr-2019)
- Improved markdown parsing performance which speeds up the final stage of exporting by around 4.5 times.
- Improved performance when exporting as PlainText or CSV.
- [GUI] Fixed an issue where the app would freeze up and report as "not responding" during the final stage of exporting.
- [CLI] Fixed an issue where it would say "Completed" after exporting a chat log, even if the process failed.
### v2.11 (14-Mar-2019)
- [HTML] Added syntax highlighting for multiline code blocks via Highlight.js.
- Added retry policy for all 5xx status codes to prevent random crashes.
### v2.10.2 (10-Mar-2019)
- [HTML] Updated message grouping algorithm to make it the same as in Discord. Removed "message group limit" setting and parameter.
- [HTML] Added small margin at the end of the chatlog so it doesn't look like it was truncated.
- [CLI] Fixed an issue where the app would crash if stdout was redirected. Progress will not be reported in such cases.
### v2.10.1 (06-Mar-2019)
- [HTML] Fixed an issue where multiple emojis on a single line would get rendered as one emoji.
+25
View File
@@ -0,0 +1,25 @@
<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2015/06/nuspec.xsd">
<metadata>
<id>discordchatexporter</id>
<title>DiscordChatExporter</title>
<version>2.12.1</version>
<owners>Tyrrrz</owners>
<authors>Tyrrrz</authors>
<copyright>Copyright (C) Alexey Golub</copyright>
<summary>Exports Discord chat logs to a file</summary>
<description>DiscordChatExporter can be used to export message history from a Discord channel to a file. It works for both direct message chats and guild chats, supports markdown, message grouping, and attachments. The tool also lets you select from/to dates to limit the exported messages. There are options to configure the output, such as date format, color theme, message grouping limit, etc.</description>
<tags>discordchatexporter discord export chat logs archive gui</tags>
<projectSourceUrl>https://github.com/Tyrrrz/DiscordChatExporter</projectSourceUrl>
<packageSourceUrl>https://github.com/Tyrrrz/DiscordChatExporter</packageSourceUrl>
<projectUrl>http://tyrrrz.me/Projects/DiscordChatExporter</projectUrl>
<releaseNotes>https://github.com/Tyrrrz/DiscordChatExporter/blob/master/Changelog.md</releaseNotes>
<iconUrl>https://raw.githubusercontent.com/Tyrrrz/DiscordChatExporter/master/favicon.png</iconUrl>
<licenseUrl>https://raw.githubusercontent.com/Tyrrrz/DiscordChatExporter/master/License.txt</licenseUrl>
<requireLicenseAcceptance>true</requireLicenseAcceptance>
<bugTrackerUrl>https://github.com/Tyrrrz/DiscordChatExporter/issues</bugTrackerUrl>
</metadata>
<files>
<file src="tools\**" target="tools" />
</files>
</package>
+24
View File
@@ -0,0 +1,24 @@
$ErrorActionPreference = 'Stop'
$packageName = $env:ChocolateyPackageName
$installDirPath = "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"
# Install package (GUI)
$packageArgs = @{
packageName = $packageName
unzipLocation = $installDirPath
url = 'https://github.com/Tyrrrz/DiscordChatExporter/releases/download/2.12.1/DiscordChatExporter.zip'
}
Install-ChocolateyZipPackage @packageArgs
# Mark the executable as GUI
New-Item (Join-Path $installDirPath "DiscordChatExporter.exe.gui") -ItemType File -Force
# Install package (CLI)
$packageArgs = @{
packageName = $packageName
unzipLocation = $installDirPath
url = 'https://github.com/Tyrrrz/DiscordChatExporter/releases/download/2.12.1/DiscordChatExporter.CLI.zip'
}
Install-ChocolateyZipPackage @packageArgs
+20 -6
View File
@@ -1,13 +1,27 @@
New-Item "$PSScriptRoot\bin" -ItemType Directory -Force
# --- PORTABLE / GUI ---
# GUI
# Get files
$files = @()
$files += Get-Item -Path "$PSScriptRoot\..\License.txt"
$files += Get-ChildItem -Path "$PSScriptRoot\..\DiscordChatExporter.Gui\bin\Release\*" -Include "*.exe", "*.dll", "*.config"
$files | Compress-Archive -DestinationPath "$PSScriptRoot\bin\DiscordChatExporter.zip" -Force
# CLI
# Pack into archive
New-Item "$PSScriptRoot\Portable\GUI\bin" -ItemType Directory -Force
$files | Compress-Archive -DestinationPath "$PSScriptRoot\Portable\GUI\bin\DiscordChatExporter.zip" -Force
# --- PORTABLE / CLI ---
# Get files
$files = @()
$files += Get-Item -Path "$PSScriptRoot\..\License.txt"
$files += Get-ChildItem -Path "$PSScriptRoot\..\DiscordChatExporter.Cli\bin\Release\net461\*" -Include "*.exe", "*.dll", "*.config"
$files | Compress-Archive -DestinationPath "$PSScriptRoot\bin\DiscordChatExporter.CLI.zip" -Force
$files += Get-ChildItem -Path "$PSScriptRoot\..\DiscordChatExporter.Cli\bin\Release\net46\*" -Include "*.exe", "*.dll", "*.config"
# Pack into archive
New-Item "$PSScriptRoot\Portable\CLI\bin" -ItemType Directory -Force
$files | Compress-Archive -DestinationPath "$PSScriptRoot\Portable\CLI\bin\DiscordChatExporter.CLI.zip" -Force
# --- CHOCOLATEY ---
# Create package
New-Item "$PSScriptRoot\Choco\bin\" -ItemType Directory -Force
choco pack $PSScriptRoot\Choco\discordchatexporter.nuspec --out $PSScriptRoot\Choco\bin\
+1 -1
View File
@@ -11,7 +11,7 @@ namespace DiscordChatExporter.Cli
{
var builder = new StyletIoCBuilder();
// Autobind services in the .Core assembly
// Autobind the .Services assembly
builder.Autobind(typeof(DataService).Assembly);
// Bind settings as singleton
@@ -1,22 +1,24 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net461</TargetFramework>
<Version>2.10.1</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Alexey Golub</Copyright>
<ApplicationIcon>..\favicon.ico</ApplicationIcon>
</PropertyGroup>
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>net46;netcoreapp2.1</TargetFrameworks>
<Version>2.12.1</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Alexey Golub</Copyright>
<ApplicationIcon>..\favicon.ico</ApplicationIcon>
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CommandLineParser" Version="2.3.0" />
<PackageReference Include="Stylet" Version="1.1.22" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.5.1" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="CommandLineParser" Version="2.3.0" />
<PackageReference Include="Stylet" Version="1.1.22" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.1" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core\DiscordChatExporter.Core.csproj" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core.Models\DiscordChatExporter.Core.Models.csproj" />
<ProjectReference Include="..\DiscordChatExporter.Core.Services\DiscordChatExporter.Core.Services.csproj" />
</ItemGroup>
</Project>
@@ -7,22 +7,41 @@ namespace DiscordChatExporter.Cli.Internal
private readonly int _posX;
private readonly int _posY;
private bool _isCompleted;
public InlineProgress()
{
_posX = Console.CursorLeft;
_posY = Console.CursorTop;
// If output is not redirected - save initial cursor position
if (!Console.IsOutputRedirected)
{
_posX = Console.CursorLeft;
_posY = Console.CursorTop;
}
}
public void Report(double progress)
{
Console.SetCursorPosition(_posX, _posY);
Console.WriteLine($"{progress:P1}");
// If output is not redirected - reset cursor position and write progress
if (!Console.IsOutputRedirected)
{
Console.SetCursorPosition(_posX, _posY);
Console.WriteLine($"{progress:P1}");
}
}
public void ReportCompletion() => _isCompleted = true;
public void Dispose()
{
Console.SetCursorPosition(_posX, _posY);
Console.WriteLine("Completed ✓");
// If output is not redirected - reset cursor position
if (!Console.IsOutputRedirected)
Console.SetCursorPosition(_posX, _posY);
// Inform about completion
if (_isCompleted)
Console.WriteLine("Completed ✓");
else
Console.WriteLine("Failed X");
}
}
}
@@ -3,8 +3,8 @@ using System.IO;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Internal;
using DiscordChatExporter.Cli.Verbs.Options;
using DiscordChatExporter.Core.Helpers;
using DiscordChatExporter.Core.Services;
using DiscordChatExporter.Core.Services.Helpers;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Cli.Verbs
@@ -24,10 +24,8 @@ namespace DiscordChatExporter.Cli.Verbs
var exportService = Container.Instance.Get<ExportService>();
// Configure settings
if (Options.DateFormat.IsNotBlank())
if (!Options.DateFormat.IsNullOrWhiteSpace())
settingsService.DateFormat = Options.DateFormat;
if (Options.MessageGroupLimit > 0)
settingsService.MessageGroupLimit = Options.MessageGroupLimit;
// Track progress
Console.Write($"Exporting channel [{Options.ChannelId}]... ");
@@ -39,7 +37,7 @@ namespace DiscordChatExporter.Cli.Verbs
// Generate file path if not set or is a directory
var filePath = Options.OutputPath;
if (filePath.IsBlank() || ExportHelper.IsDirectoryPath(filePath))
if (filePath.IsNullOrWhiteSpace() || ExportHelper.IsDirectoryPath(filePath))
{
// Generate default file name
var fileName = ExportHelper.GetDefaultExportFileName(Options.ExportFormat, chatLog.Guild,
@@ -50,7 +48,10 @@ namespace DiscordChatExporter.Cli.Verbs
}
// Export
exportService.ExportChatLog(chatLog, filePath, Options.ExportFormat, Options.PartitionLimit);
await exportService.ExportChatLogAsync(chatLog, filePath, Options.ExportFormat, Options.PartitionLimit);
// Report successful completion
progress.ReportCompletion();
}
}
}
@@ -5,9 +5,9 @@ using System.Net;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Internal;
using DiscordChatExporter.Cli.Verbs.Options;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Helpers;
using DiscordChatExporter.Core.Services;
using DiscordChatExporter.Core.Services.Exceptions;
using DiscordChatExporter.Core.Services.Helpers;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Cli.Verbs
@@ -27,10 +27,8 @@ namespace DiscordChatExporter.Cli.Verbs
var exportService = Container.Instance.Get<ExportService>();
// Configure settings
if (Options.DateFormat.IsNotBlank())
if (!Options.DateFormat.IsNullOrWhiteSpace())
settingsService.DateFormat = Options.DateFormat;
if (Options.MessageGroupLimit > 0)
settingsService.MessageGroupLimit = Options.MessageGroupLimit;
// Get channels
var channels = await dataService.GetDirectMessageChannelsAsync(Options.GetToken());
@@ -59,7 +57,11 @@ namespace DiscordChatExporter.Cli.Verbs
var filePath = Path.Combine(Options.OutputPath ?? "", fileName);
// Export
exportService.ExportChatLog(chatLog, filePath, Options.ExportFormat, Options.PartitionLimit);
await exportService.ExportChatLogAsync(chatLog, filePath, Options.ExportFormat,
Options.PartitionLimit);
// Report successful completion
progress.ReportCompletion();
}
}
catch (HttpErrorStatusCodeException ex) when (ex.StatusCode == HttpStatusCode.Forbidden)
@@ -5,10 +5,10 @@ using System.Net;
using System.Threading.Tasks;
using DiscordChatExporter.Cli.Internal;
using DiscordChatExporter.Cli.Verbs.Options;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Helpers;
using DiscordChatExporter.Core.Models;
using DiscordChatExporter.Core.Services;
using DiscordChatExporter.Core.Services.Exceptions;
using DiscordChatExporter.Core.Services.Helpers;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Cli.Verbs
@@ -28,10 +28,8 @@ namespace DiscordChatExporter.Cli.Verbs
var exportService = Container.Instance.Get<ExportService>();
// Configure settings
if (Options.DateFormat.IsNotBlank())
if (!Options.DateFormat.IsNullOrWhiteSpace())
settingsService.DateFormat = Options.DateFormat;
if (Options.MessageGroupLimit > 0)
settingsService.MessageGroupLimit = Options.MessageGroupLimit;
// Get channels
var channels = await dataService.GetGuildChannelsAsync(Options.GetToken(), Options.GuildId);
@@ -60,7 +58,11 @@ namespace DiscordChatExporter.Cli.Verbs
var filePath = Path.Combine(Options.OutputPath ?? "", fileName);
// Export
exportService.ExportChatLog(chatLog, filePath, Options.ExportFormat, Options.PartitionLimit);
await exportService.ExportChatLogAsync(chatLog, filePath, Options.ExportFormat,
Options.PartitionLimit);
// Report successful completion
progress.ReportCompletion();
}
}
catch (HttpErrorStatusCodeException ex) when (ex.StatusCode == HttpStatusCode.Forbidden)
@@ -12,9 +12,11 @@ namespace DiscordChatExporter.Cli.Verbs.Options
[Option('o', "output", Default = null, HelpText = "Output file or directory path.")]
public string OutputPath { get; set; }
// HACK: CommandLineParser doesn't support DateTimeOffset
[Option("after", Default = null, HelpText = "Limit to messages sent after this date.")]
public DateTime? After { get; set; }
// HACK: CommandLineParser doesn't support DateTimeOffset
[Option("before", Default = null, HelpText = "Limit to messages sent before this date.")]
public DateTime? Before { get; set; }
@@ -23,8 +25,5 @@ namespace DiscordChatExporter.Cli.Verbs.Options
[Option("dateformat", Default = null, HelpText = "Date format used in output.")]
public string DateFormat { get; set; }
[Option("grouplimit", Default = 0, HelpText = "Message group limit.")]
public int MessageGroupLimit { get; set; }
}
}
@@ -1,12 +1,11 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net461</TargetFramework>
<TargetFrameworks>net46;netstandard2.0</TargetFrameworks>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Sprache" Version="2.2.0" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.5.1" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.1" />
</ItemGroup>
</Project>
</Project>
@@ -0,0 +1,46 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown.Internal
{
internal class AggregateMatcher<T> : IMatcher<T>
{
private readonly IReadOnlyList<IMatcher<T>> _matchers;
public AggregateMatcher(IReadOnlyList<IMatcher<T>> matchers)
{
_matchers = matchers;
}
public AggregateMatcher(params IMatcher<T>[] matchers)
: this((IReadOnlyList<IMatcher<T>>)matchers)
{
}
public ParsedMatch<T> Match(string input, int startIndex, int length)
{
ParsedMatch<T> earliestMatch = null;
// Try to match the input with each matcher and get the match with the lowest start index
foreach (var matcher in _matchers)
{
// Try to match
var match = matcher.Match(input, startIndex, length);
// If there's no match - continue
if (match == null)
continue;
// If this match is earlier than previous earliest - replace
if (earliestMatch == null || match.StartIndex < earliestMatch.StartIndex)
earliestMatch = match;
// If the earliest match starts at the very beginning - break,
// because it's impossible to find a match earlier than that
if (earliestMatch.StartIndex == startIndex)
break;
}
return earliestMatch;
}
}
}
@@ -0,0 +1,50 @@
using System;
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown.Internal
{
internal static class Extensions
{
public static IEnumerable<ParsedMatch<T>> MatchAll<T>(this IMatcher<T> matcher, string input,
int startIndex, int length, Func<string, T> fallbackTransform)
{
// Get end index for simplicity
var endIndex = startIndex + length;
// Loop through segments divided by individual matches
var currentIndex = startIndex;
while (currentIndex < endIndex)
{
// Find a match within this segment
var match = matcher.Match(input, currentIndex, endIndex - currentIndex);
// If there's no match - break
if (match == null)
break;
// If this match doesn't start immediately at current index - transform and yield fallback first
if (match.StartIndex > currentIndex)
{
var fallback = input.Substring(currentIndex, match.StartIndex - currentIndex);
yield return new ParsedMatch<T>(currentIndex, fallback.Length, fallbackTransform(fallback));
}
// Yield match
yield return match;
// Shift current index to the end of the match
currentIndex = match.StartIndex + match.Length;
}
// If EOL wasn't reached - transform and yield remaining part as fallback
if (currentIndex < endIndex)
{
var fallback = input.Substring(currentIndex);
yield return new ParsedMatch<T>(currentIndex, fallback.Length, fallbackTransform(fallback));
}
}
public static IEnumerable<ParsedMatch<T>> MatchAll<T>(this IMatcher<T> matcher, string input,
Func<string, T> fallbackTransform) => matcher.MatchAll(input, 0, input.Length, fallbackTransform);
}
}
@@ -1,178 +0,0 @@
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
using Sprache;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Markdown.Internal
{
// The following parsing logic is meant to replicate Discord's markdown grammar as close as possible
internal static class Grammar
{
/* Formatting */
// Capture until the earliest double asterisk not followed by an asterisk
private static readonly Parser<Node> BoldFormattedNode =
Parse.RegexMatch(new Regex("\\*\\*(.+?)\\*\\*(?!\\*)", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "**", TextFormatting.Bold, BuildTree(m.Groups[1].Value)));
// Capture until the earliest single asterisk not preceded or followed by an asterisk
// Can't have whitespace right after opening or right before closing asterisk
private static readonly Parser<Node> ItalicFormattedNode =
Parse.RegexMatch(new Regex("\\*(?!\\s)(.+?)(?<!\\s|\\*)\\*(?!\\*)", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "*", TextFormatting.Italic, BuildTree(m.Groups[1].Value)));
// Can't have underscores inside
// Can't have word characters right after closing underscore
private static readonly Parser<Node> ItalicAltFormattedNode =
Parse.RegexMatch(new Regex("_([^_]+?)_(?!\\w)", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "_", TextFormatting.Italic, BuildTree(m.Groups[1].Value)));
// Treated as a separate entity for simplicity
// Capture until the earliest triple asterisk not preceded or followed by an asterisk
private static readonly Parser<Node> ItalicBoldFormattedNode =
Parse.RegexMatch(new Regex("\\*(\\*\\*(?:.+?)\\*\\*)\\*(?!\\*)", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "*", TextFormatting.Italic, BuildTree(m.Groups[1].Value)));
// Capture until the earliest double underscore not followed by an underscore
private static readonly Parser<Node> UnderlineFormattedNode =
Parse.RegexMatch(new Regex("__(.+?)__(?!_)", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "__", TextFormatting.Underline, BuildTree(m.Groups[1].Value)));
// Treated as a separate entity for simplicity
// Capture until the earliest triple underscore not preceded or followed by an underscore
private static readonly Parser<Node> ItalicUnderlineFormattedNode =
Parse.RegexMatch(new Regex("_(__(?:.+?)__)_(?!_)", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "_", TextFormatting.Italic, BuildTree(m.Groups[1].Value)));
// Strikethrough is safe
private static readonly Parser<Node> StrikethroughFormattedNode =
Parse.RegexMatch(new Regex("~~(.+?)~~", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "~~", TextFormatting.Strikethrough, BuildTree(m.Groups[1].Value)));
// Spoiler is safe
private static readonly Parser<Node> SpoilerFormattedNode =
Parse.RegexMatch(new Regex("\\|\\|(.+?)\\|\\|", RegexOptions.Singleline))
.Select(m => new FormattedNode(m.Value, "||", TextFormatting.Spoiler, BuildTree(m.Groups[1].Value)));
// Combinator, order matters
private static readonly Parser<Node> AnyFormattedNode =
ItalicBoldFormattedNode.Or(ItalicUnderlineFormattedNode)
.Or(BoldFormattedNode).Or(ItalicFormattedNode)
.Or(UnderlineFormattedNode).Or(ItalicAltFormattedNode)
.Or(StrikethroughFormattedNode).Or(SpoilerFormattedNode);
/* Code blocks */
// Can't have backticks inside and surrounding whitespace is trimmed
private static readonly Parser<Node> InlineCodeBlockNode =
Parse.RegexMatch(new Regex("`\\s*([^`]+?)\\s*`", RegexOptions.Singleline))
.Select(m => new InlineCodeBlockNode(m.Value, m.Groups[1].Value));
// The first word is a language identifier if it's the only word followed by a newline, the rest is code
private static readonly Parser<Node> MultilineCodeBlockNode =
Parse.RegexMatch(new Regex("```(?:(\\w*?)?(?:\\s*?\\n))?(.+?)```", RegexOptions.Singleline))
.Select(m => new MultilineCodeBlockNode(m.Value, m.Groups[1].Value, m.Groups[2].Value));
// Combinator, order matters
private static readonly Parser<Node> AnyCodeBlockNode = MultilineCodeBlockNode.Or(InlineCodeBlockNode);
/* Mentions */
// @everyone or @here
private static readonly Parser<Node> MetaMentionNode = Parse.RegexMatch("@(everyone|here)")
.Select(m => new MentionNode(m.Value, m.Groups[1].Value, MentionType.Meta));
// <@123456> or <@!123456>
private static readonly Parser<Node> UserMentionNode = Parse.RegexMatch("<@!?(\\d+)>")
.Select(m => new MentionNode(m.Value, m.Groups[1].Value, MentionType.User));
// <#123456>
private static readonly Parser<Node> ChannelMentionNode = Parse.RegexMatch("<#(\\d+)>")
.Select(m => new MentionNode(m.Value, m.Groups[1].Value, MentionType.Channel));
// <@&123456>
private static readonly Parser<Node> RoleMentionNode = Parse.RegexMatch("<@&(\\d+)>")
.Select(m => new MentionNode(m.Value, m.Groups[1].Value, MentionType.Role));
// Combinator, order matters
private static readonly Parser<Node> AnyMentionNode =
MetaMentionNode.Or(UserMentionNode).Or(ChannelMentionNode).Or(RoleMentionNode);
/* Emojis */
// Matches all standard unicode emojis
private static readonly Parser<Node> StandardEmojiNode = Parse.RegexMatch(
"([\\u2700-\\u27bf]|" +
"(?:\\ud83c[\\udde6-\\uddff]){2}|" +
"[\\ud800-\\udbff][\\udc00-\\udfff]|" +
"[\\u0023-\\u0039]\\u20e3|" +
"\\u3299|\\u3297|\\u303d|\\u3030|\\u24c2|\\ud83c[\\udd70-\\udd71]|\\ud83c[\\udd7e-\\udd7f]|\\ud83c\\udd8e|\\ud83c[\\udd91-\\udd9a]|\\ud83c[\\udde6-\\uddff]|" +
"\\ud83c[\\ude01-\\ude02]|\\ud83c\\ude1a|\\ud83c\\ude2f|\\ud83c[\\ude32-\\ude3a]|\\ud83c[\\ude50-\\ude51]|\\u203c|\\u2049|[\\u25aa-\\u25ab]|" +
"\\u25b6|\\u25c0|[\\u25fb-\\u25fe]|\\u00a9|\\u00ae|\\u2122|\\u2139|\\ud83c\\udc04|[\\u2600-\\u26FF]|\\u2b05|\\u2b06|\\u2b07|\\u2b1b|\\u2b1c|\\u2b50|" +
"\\u2b55|\\u231a|\\u231b|\\u2328|\\u23cf|[\\u23e9-\\u23f3]|[\\u23f8-\\u23fa]|\\ud83c\\udccf|\\u2934|\\u2935|[\\u2190-\\u21ff])")
.Select(m => new EmojiNode(m.Value, m.Groups[1].Value));
// <:lul:123456> or <a:lul:123456>
private static readonly Parser<Node> CustomEmojiNode = Parse.RegexMatch("<(a)?:(.+?):(\\d+)>")
.Select(m => new EmojiNode(m.Value, m.Groups[3].Value, m.Groups[2].Value, m.Groups[1].Value.IsNotBlank()));
// Combinator, order matters
private static readonly Parser<Node> AnyEmojiNode = StandardEmojiNode.Or(CustomEmojiNode);
/* Links */
// [title](link)
private static readonly Parser<Node> TitledLinkNode = Parse.RegexMatch("\\[(.+?)\\]\\((.+?)\\)")
.Select(m => new LinkNode(m.Value, m.Groups[2].Value, m.Groups[1].Value));
// Starts with http:// or https://, stops at the last non-whitespace character followed by whitespace or punctuation character
private static readonly Parser<Node> AutoLinkNode = Parse.RegexMatch("(https?://\\S*[^\\.,:;\"\'\\s])")
.Select(m => new LinkNode(m.Value, m.Groups[1].Value));
// Autolink surrounded by angular brackets
private static readonly Parser<Node> HiddenLinkNode = Parse.RegexMatch("<(https?://\\S*[^\\.,:;\"\'\\s])>")
.Select(m => new LinkNode(m.Value, m.Groups[1].Value));
// Combinator, order matters
private static readonly Parser<Node> AnyLinkNode = TitledLinkNode.Or(HiddenLinkNode).Or(AutoLinkNode);
/* Text */
// Shrug is an exception and needs to be exempt from formatting
private static readonly Parser<Node> ShrugTextNode =
Parse.String("¯\\_(ツ)_/¯").Text().Select(s => new TextNode(s));
// Backslash escapes any following unicode surrogate pair
private static readonly Parser<Node> EscapedSurrogateTextNode =
from slash in Parse.Char('\\')
from high in Parse.AnyChar.Where(char.IsHighSurrogate)
from low in Parse.AnyChar
let lexeme = $"{slash}{high}{low}"
let text = $"{high}{low}"
select new TextNode(lexeme, text);
// Backslash escapes any following non-whitespace character except for digits and latin letters
private static readonly Parser<Node> EscapedTextNode =
Parse.RegexMatch("\\\\([^a-zA-Z0-9\\s])").Select(m => new TextNode(m.Value, m.Groups[1].Value));
// Combinator, order matters
private static readonly Parser<Node> AnyTextNode = ShrugTextNode.Or(EscapedSurrogateTextNode).Or(EscapedTextNode);
/* Aggregator and fallback */
// Any node recognized by above patterns
private static readonly Parser<Node> AnyRecognizedNode = AnyFormattedNode.Or(AnyCodeBlockNode)
.Or(AnyMentionNode).Or(AnyEmojiNode).Or(AnyLinkNode).Or(AnyTextNode);
// Any node not recognized by above patterns (treated as plain text)
private static readonly Parser<Node> FallbackNode =
Parse.AnyChar.Except(AnyRecognizedNode).AtLeastOnce().Text().Select(s => new TextNode(s));
// Any node
private static readonly Parser<Node> AnyNode = AnyRecognizedNode.Or(FallbackNode);
// Entry point
public static IReadOnlyList<Node> BuildTree(string input) => AnyNode.Many().Parse(input).ToArray();
}
}
@@ -0,0 +1,7 @@
namespace DiscordChatExporter.Core.Markdown.Internal
{
internal interface IMatcher<T>
{
ParsedMatch<T> Match(string input, int startIndex, int length);
}
}
@@ -0,0 +1,18 @@
namespace DiscordChatExporter.Core.Markdown.Internal
{
internal partial class ParsedMatch<T>
{
public int StartIndex { get; }
public int Length { get; }
public T Value { get; }
public ParsedMatch(int startIndex, int length, T value)
{
StartIndex = startIndex;
Length = length;
Value = value;
}
}
}
@@ -0,0 +1,23 @@
using System;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Markdown.Internal
{
internal class RegexMatcher<T> : IMatcher<T>
{
private readonly Regex _regex;
private readonly Func<Match, T> _transform;
public RegexMatcher(Regex regex, Func<Match, T> transform)
{
_regex = regex;
_transform = transform;
}
public ParsedMatch<T> Match(string input, int startIndex, int length)
{
var match = _regex.Match(input, startIndex, length);
return match.Success ? new ParsedMatch<T>(match.Index, match.Length, _transform(match)) : null;
}
}
}
@@ -0,0 +1,29 @@
using System;
namespace DiscordChatExporter.Core.Markdown.Internal
{
internal class StringMatcher<T> : IMatcher<T>
{
private readonly string _needle;
private readonly StringComparison _comparison;
private readonly Func<string, T> _transform;
public StringMatcher(string needle, StringComparison comparison, Func<string, T> transform)
{
_needle = needle;
_comparison = comparison;
_transform = transform;
}
public StringMatcher(string needle, Func<string, T> transform)
: this(needle, StringComparison.Ordinal, transform)
{
}
public ParsedMatch<T> Match(string input, int startIndex, int length)
{
var index = input.IndexOf(_needle, startIndex, length, _comparison);
return index >= 0 ? new ParsedMatch<T>(index, _needle.Length, _transform(_needle)) : null;
}
}
}
@@ -1,10 +1,205 @@
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Markdown.Internal;
using DiscordChatExporter.Core.Markdown.Nodes;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Markdown
{
// The following parsing logic is meant to replicate Discord's markdown grammar as close as possible
public static class MarkdownParser
{
public static IReadOnlyList<Node> Parse(string input) => Grammar.BuildTree(input);
private const RegexOptions DefaultRegexOptions = RegexOptions.Compiled | RegexOptions.CultureInvariant;
/* Formatting */
// Capture any character until the earliest double asterisk not followed by an asterisk
private static readonly IMatcher<Node> BoldFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("\\*\\*(.+?)\\*\\*(?!\\*)", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "**", TextFormatting.Bold, Parse(m.Groups[1].Value)));
// Capture any character until the earliest single asterisk not preceded or followed by an asterisk
// Opening asterisk must not be followed by whitespace
// Closing asterisk must not be preceeded by whitespace
private static readonly IMatcher<Node> ItalicFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("\\*(?!\\s)(.+?)(?<!\\s|\\*)\\*(?!\\*)", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "*", TextFormatting.Italic, Parse(m.Groups[1].Value)));
// Capture any character until the earliest triple asterisk not followed by an asterisk
private static readonly IMatcher<Node> ItalicBoldFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("\\*(\\*\\*.+?\\*\\*)\\*(?!\\*)", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "*", TextFormatting.Italic, Parse(m.Groups[1].Value, BoldFormattedNodeMatcher)));
// Capture any character except underscore until an underscore
// Closing underscore must not be followed by a word character
private static readonly IMatcher<Node> ItalicAltFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("_([^_]+)_(?!\\w)", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "_", TextFormatting.Italic, Parse(m.Groups[1].Value)));
// Capture any character until the earliest double underscore not followed by an underscore
private static readonly IMatcher<Node> UnderlineFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("__(.+?)__(?!_)", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "__", TextFormatting.Underline, Parse(m.Groups[1].Value)));
// Capture any character until the earliest triple underscore not followed by an underscore
private static readonly IMatcher<Node> ItalicUnderlineFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("_(__.+?__)_(?!_)", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "_", TextFormatting.Italic, Parse(m.Groups[1].Value, UnderlineFormattedNodeMatcher)));
// Capture any character until the earliest double tilde
private static readonly IMatcher<Node> StrikethroughFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("~~(.+?)~~", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "~~", TextFormatting.Strikethrough, Parse(m.Groups[1].Value)));
// Capture any character until the earliest double pipe
private static readonly IMatcher<Node> SpoilerFormattedNodeMatcher = new RegexMatcher<Node>(
new Regex("\\|\\|(.+?)\\|\\|", DefaultRegexOptions | RegexOptions.Singleline),
m => new FormattedNode(m.Value, "||", TextFormatting.Spoiler, Parse(m.Groups[1].Value)));
/* Code blocks */
// Capture any character except backtick until a backtick
// Whitespace surrounding content inside backticks is trimmed
private static readonly IMatcher<Node> InlineCodeBlockNodeMatcher = new RegexMatcher<Node>(
new Regex("`([^`]+)`", DefaultRegexOptions | RegexOptions.Singleline),
m => new InlineCodeBlockNode(m.Value, m.Groups[1].Value.Trim()));
// Capture language identifier and then any character until the earliest triple backtick
// Languge identifier is one word immediately after opening backticks, followed immediately by newline
// Whitespace surrounding content inside backticks is trimmed
private static readonly IMatcher<Node> MultilineCodeBlockNodeMatcher = new RegexMatcher<Node>(
new Regex("```(?:(\\w*)\\n)?(.+?)```", DefaultRegexOptions | RegexOptions.Singleline),
m => new MultilineCodeBlockNode(m.Value, m.Groups[1].Value, m.Groups[2].Value.Trim()));
/* Mentions */
// Capture @everyone
private static readonly IMatcher<Node> EveryoneMentionNodeMatcher = new StringMatcher<Node>(
"@everyone",
s => new MentionNode(s, "everyone", MentionType.Meta));
// Capture @here
private static readonly IMatcher<Node> HereMentionNodeMatcher = new StringMatcher<Node>(
"@here",
s => new MentionNode(s, "here", MentionType.Meta));
// Capture <@123456> or <@!123456>
private static readonly IMatcher<Node> UserMentionNodeMatcher = new RegexMatcher<Node>(
new Regex("<@!?(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Value, m.Groups[1].Value, MentionType.User));
// Capture <#123456>
private static readonly IMatcher<Node> ChannelMentionNodeMatcher = new RegexMatcher<Node>(
new Regex("<#(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Value, m.Groups[1].Value, MentionType.Channel));
// Capture <@&123456>
private static readonly IMatcher<Node> RoleMentionNodeMatcher = new RegexMatcher<Node>(
new Regex("<@&(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Value, m.Groups[1].Value, MentionType.Role));
/* Emojis */
// Capture any country flag emoji (two regional indicator surrogate pairs)
// ... or "symbol/other" character
// ... or surrogate pair
// ... or digit followed by enclosing mark
// (this does not match all emojis in Discord but it's reasonably accurate enough)
private static readonly IMatcher<Node> StandardEmojiNodeMatcher = new RegexMatcher<Node>(
new Regex("((?:[\\uD83C][\\uDDE6-\\uDDFF]){2}|\\p{So}|\\p{Cs}{2}|\\d\\p{Me})", DefaultRegexOptions),
m => new EmojiNode(m.Value, m.Groups[1].Value));
// Capture <:lul:123456> or <a:lul:123456>
private static readonly IMatcher<Node> CustomEmojiNodeMatcher = new RegexMatcher<Node>(
new Regex("<(a)?:(.+?):(\\d+?)>", DefaultRegexOptions),
m => new EmojiNode(m.Value, m.Groups[3].Value, m.Groups[2].Value, !m.Groups[1].Value.IsNullOrWhiteSpace()));
/* Links */
// Capture [title](link)
private static readonly IMatcher<Node> TitledLinkNodeMatcher = new RegexMatcher<Node>(
new Regex("\\[(.+?)\\]\\((.+?)\\)", DefaultRegexOptions),
m => new LinkNode(m.Value, m.Groups[2].Value, m.Groups[1].Value));
// Capture any non-whitespace character after http:// or https:// until the last punctuation character or whitespace
private static readonly IMatcher<Node> AutoLinkNodeMatcher = new RegexMatcher<Node>(
new Regex("(https?://\\S*[^\\.,:;\"\'\\s])", DefaultRegexOptions),
m => new LinkNode(m.Value, m.Groups[1].Value));
// Same as auto link but also surrounded by angular brackets
private static readonly IMatcher<Node> HiddenLinkNodeMatcher = new RegexMatcher<Node>(
new Regex("<(https?://\\S*[^\\.,:;\"\'\\s])>", DefaultRegexOptions),
m => new LinkNode(m.Value, m.Groups[1].Value));
/* Text */
// Capture the shrug emoticon
// This escapes it from matching for formatting
private static readonly IMatcher<Node> ShrugTextNodeMatcher = new StringMatcher<Node>(
@"¯\_(ツ)_/¯",
s => new TextNode(s));
// Capture some specific emojis that don't get rendered
// This escapes it from matching for emoji
private static readonly IMatcher<Node> IgnoredEmojiTextNodeMatcher = new RegexMatcher<Node>(
new Regex("(\\u26A7|\\u2640|\\u2642|\\u2695|\\u267E|\\u00A9|\\u00AE|\\u2122)", DefaultRegexOptions),
m => new TextNode(m.Value, m.Groups[1].Value));
// Capture any "symbol/other" character or surrogate pair preceeded by a backslash
// This escapes it from matching for emoji
private static readonly IMatcher<Node> EscapedSymbolTextNodeMatcher = new RegexMatcher<Node>(
new Regex("\\\\(\\p{So}|\\p{Cs}{2})", DefaultRegexOptions),
m => new TextNode(m.Value, m.Groups[1].Value));
// Capture any non-whitespace, non latin alphanumeric character preceeded by a backslash
// This escapes it from matching for formatting or other tokens
private static readonly IMatcher<Node> EscapedCharacterTextNodeMatcher = new RegexMatcher<Node>(
new Regex("\\\\([^a-zA-Z0-9\\s])", DefaultRegexOptions),
m => new TextNode(m.Value, m.Groups[1].Value));
// Combine all matchers into one
// Matchers that have similar patterns are ordered from most specific to least specific
private static readonly IMatcher<Node> AggregateNodeMatcher = new AggregateMatcher<Node>(
// Escaped text
ShrugTextNodeMatcher,
IgnoredEmojiTextNodeMatcher,
EscapedSymbolTextNodeMatcher,
EscapedCharacterTextNodeMatcher,
// Formatting
ItalicBoldFormattedNodeMatcher,
ItalicUnderlineFormattedNodeMatcher,
BoldFormattedNodeMatcher,
ItalicFormattedNodeMatcher,
UnderlineFormattedNodeMatcher,
ItalicAltFormattedNodeMatcher,
StrikethroughFormattedNodeMatcher,
SpoilerFormattedNodeMatcher,
// Code blocks
MultilineCodeBlockNodeMatcher,
InlineCodeBlockNodeMatcher,
// Mentions
EveryoneMentionNodeMatcher,
HereMentionNodeMatcher,
UserMentionNodeMatcher,
ChannelMentionNodeMatcher,
RoleMentionNodeMatcher,
// Links
TitledLinkNodeMatcher,
AutoLinkNodeMatcher,
HiddenLinkNodeMatcher,
// Emoji
StandardEmojiNodeMatcher,
CustomEmojiNodeMatcher);
private static IReadOnlyList<Node> Parse(string input, IMatcher<Node> matcher) =>
matcher.MatchAll(input, s => new TextNode(s)).Select(r => r.Value).ToArray();
public static IReadOnlyList<Node> Parse(string input) => Parse(input, AggregateNodeMatcher);
}
}
-12
View File
@@ -1,12 +0,0 @@
namespace DiscordChatExporter.Core.Markdown
{
public abstract class Node
{
public string Lexeme { get; }
protected Node(string lexeme)
{
Lexeme = lexeme;
}
}
}
@@ -1,6 +1,6 @@
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public class EmojiNode : Node
{
@@ -10,18 +10,18 @@ namespace DiscordChatExporter.Core.Markdown
public bool IsAnimated { get; }
public bool IsCustomEmoji => Id.IsNotBlank();
public bool IsCustomEmoji => !Id.IsNullOrWhiteSpace();
public EmojiNode(string lexeme, string id, string name, bool isAnimated)
: base(lexeme)
public EmojiNode(string source, string id, string name, bool isAnimated)
: base(source)
{
Id = id;
Name = name;
IsAnimated = isAnimated;
}
public EmojiNode(string lexeme, string name)
: this(lexeme, null, name, false)
public EmojiNode(string source, string name)
: this(source, null, name, false)
{
}
@@ -1,6 +1,6 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public class FormattedNode : Node
{
@@ -10,8 +10,8 @@ namespace DiscordChatExporter.Core.Markdown
public IReadOnlyList<Node> Children { get; }
public FormattedNode(string lexeme, string token, TextFormatting formatting, IReadOnlyList<Node> children)
: base(lexeme)
public FormattedNode(string source, string token, TextFormatting formatting, IReadOnlyList<Node> children)
: base(source)
{
Token = token;
Formatting = formatting;
@@ -1,11 +1,11 @@
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public class InlineCodeBlockNode : Node
{
public string Code { get; }
public InlineCodeBlockNode(string lexeme, string code)
: base(lexeme)
public InlineCodeBlockNode(string source, string code)
: base(source)
{
Code = code;
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public class LinkNode : Node
{
@@ -6,14 +6,14 @@
public string Title { get; }
public LinkNode(string lexeme, string url, string title)
: base(lexeme)
public LinkNode(string source, string url, string title)
: base(source)
{
Url = url;
Title = title;
}
public LinkNode(string lexeme, string url) : this(lexeme, url, url)
public LinkNode(string source, string url) : this(source, url, url)
{
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public class MentionNode : Node
{
@@ -6,8 +6,8 @@
public MentionType Type { get; }
public MentionNode(string lexeme, string id, MentionType type)
: base(lexeme)
public MentionNode(string source, string id, MentionType type)
: base(source)
{
Id = id;
Type = type;
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public enum MentionType
{
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public class MultilineCodeBlockNode : Node
{
@@ -6,8 +6,8 @@
public string Code { get; }
public MultilineCodeBlockNode(string lexeme, string language, string code)
: base(lexeme)
public MultilineCodeBlockNode(string source, string language, string code)
: base(source)
{
Language = language;
Code = code;
@@ -0,0 +1,12 @@
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public abstract class Node
{
public string Source { get; }
protected Node(string source)
{
Source = source;
}
}
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public enum TextFormatting
{
@@ -1,11 +1,11 @@
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Nodes
{
public class TextNode : Node
{
public string Text { get; }
public TextNode(string lexeme, string text)
: base(lexeme)
public TextNode(string source, string text)
: base(source)
{
Text = text;
}
@@ -1,11 +1,12 @@
using System;
using ByteSizeLib;
using System.IO;
using System.Linq;
namespace DiscordChatExporter.Core.Models
{
// https://discordapp.com/developers/docs/resources/channel#attachment-object
public class Attachment
public partial class Attachment
{
public string Id { get; }
@@ -17,15 +18,11 @@ namespace DiscordChatExporter.Core.Models
public string FileName { get; }
public bool IsImage => FileName.EndsWith(".jpg", StringComparison.OrdinalIgnoreCase) ||
FileName.EndsWith(".jpeg", StringComparison.OrdinalIgnoreCase) ||
FileName.EndsWith(".png", StringComparison.OrdinalIgnoreCase) ||
FileName.EndsWith(".gif", StringComparison.OrdinalIgnoreCase) ||
FileName.EndsWith(".bmp", StringComparison.OrdinalIgnoreCase);
public bool IsImage { get; }
public ByteSize FileSize { get; }
public FileSize FileSize { get; }
public Attachment(string id, int? width, int? height, string url, string fileName, ByteSize fileSize)
public Attachment(string id, int? width, int? height, string url, string fileName, FileSize fileSize)
{
Id = id;
Url = url;
@@ -33,8 +30,21 @@ namespace DiscordChatExporter.Core.Models
Height = height;
FileName = fileName;
FileSize = fileSize;
IsImage = GetIsImage(fileName);
}
public override string ToString() => FileName;
}
public partial class Attachment
{
private static readonly string[] ImageFileExtensions = { ".jpg", ".jpeg", ".png", ".gif", ".bmp" };
public static bool GetIsImage(string fileName)
{
var fileExtension = Path.GetExtension(fileName);
return ImageFileExtensions.Contains(fileExtension, StringComparer.OrdinalIgnoreCase);
}
}
}
@@ -9,21 +9,21 @@ namespace DiscordChatExporter.Core.Models
public Channel Channel { get; }
public DateTime? From { get; }
public DateTimeOffset? After { get; }
public DateTime? To { get; }
public DateTimeOffset? Before { get; }
public IReadOnlyList<Message> Messages { get; }
public Mentionables Mentionables { get; }
public ChatLog(Guild guild, Channel channel, DateTime? from, DateTime? to,
public ChatLog(Guild guild, Channel channel, DateTimeOffset? after, DateTimeOffset? before,
IReadOnlyList<Message> messages, Mentionables mentionables)
{
Guild = guild;
Channel = channel;
From = from;
To = to;
After = after;
Before = before;
Messages = messages;
Mentionables = mentionables;
}
@@ -0,0 +1,11 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net46;netstandard2.0</TargetFrameworks>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.1" />
</ItemGroup>
</Project>
@@ -12,7 +12,7 @@ namespace DiscordChatExporter.Core.Models
public string Url { get; }
public DateTime? Timestamp { get; }
public DateTimeOffset? Timestamp { get; }
public Color Color { get; }
@@ -28,7 +28,7 @@ namespace DiscordChatExporter.Core.Models
public EmbedFooter Footer { get; }
public Embed(string title, string url, DateTime? timestamp, Color color, EmbedAuthor author, string description,
public Embed(string title, string url, DateTimeOffset? timestamp, Color color, EmbedAuthor author, string description,
IReadOnlyList<EmbedField> fields, EmbedImage thumbnail, EmbedImage image, EmbedFooter footer)
{
Title = title;
@@ -14,31 +14,15 @@ namespace DiscordChatExporter.Core.Models
public bool IsAnimated { get; }
public string ImageUrl
{
get
{
// Custom emoji
if (Id.IsNotBlank())
{
// Animated
if (IsAnimated)
return $"https://cdn.discordapp.com/emojis/{Id}.gif";
// Non-animated
return $"https://cdn.discordapp.com/emojis/{Id}.png";
}
// Standard unicode emoji (via twemoji)
return $"https://twemoji.maxcdn.com/2/72x72/{GetTwemojiName(Name)}.png";
}
}
public string ImageUrl { get; }
public Emoji(string id, string name, bool isAnimated)
{
Id = id;
Name = name;
IsAnimated = isAnimated;
ImageUrl = GetImageUrl(id, name, isAnimated);
}
}
@@ -50,7 +34,25 @@ namespace DiscordChatExporter.Core.Models
yield return char.ConvertToUtf32(emoji, i);
}
private static string GetTwemojiName(string emoji)
=> GetCodePoints(emoji).Select(i => i.ToString("x")).JoinToString("-");
private static string GetTwemojiName(string emoji) =>
GetCodePoints(emoji).Select(i => i.ToString("x")).JoinToString("-");
public static string GetImageUrl(string id, string name, bool isAnimated)
{
// Custom emoji
if (!id.IsNullOrWhiteSpace())
{
// Animated
if (isAnimated)
return $"https://cdn.discordapp.com/emojis/{id}.gif";
// Non-animated
return $"https://cdn.discordapp.com/emojis/{id}.png";
}
// Standard unicode emoji (via twemoji)
var twemojiName = GetTwemojiName(name);
return $"https://twemoji.maxcdn.com/2/72x72/{twemojiName}.png";
}
}
}
@@ -0,0 +1,84 @@
using System;
namespace DiscordChatExporter.Core.Models
{
// Loosely based on https://github.com/omar/ByteSize (MIT license)
public struct FileSize
{
public const long BytesInKiloByte = 1024;
public const long BytesInMegaByte = 1024 * BytesInKiloByte;
public const long BytesInGigaByte = 1024 * BytesInMegaByte;
public const long BytesInTeraByte = 1024 * BytesInGigaByte;
public const long BytesInPetaByte = 1024 * BytesInTeraByte;
public const string ByteSymbol = "B";
public const string KiloByteSymbol = "KB";
public const string MegaByteSymbol = "MB";
public const string GigaByteSymbol = "GB";
public const string TeraByteSymbol = "TB";
public const string PetaByteSymbol = "PB";
public double Bytes { get; }
public double KiloBytes => Bytes / BytesInKiloByte;
public double MegaBytes => Bytes / BytesInMegaByte;
public double GigaBytes => Bytes / BytesInGigaByte;
public double TeraBytes => Bytes / BytesInTeraByte;
public double PetaBytes => Bytes / BytesInPetaByte;
public string LargestWholeNumberSymbol
{
get
{
// Absolute value is used to deal with negative values
if (Math.Abs(PetaBytes) >= 1)
return PetaByteSymbol;
if (Math.Abs(TeraBytes) >= 1)
return TeraByteSymbol;
if (Math.Abs(GigaBytes) >= 1)
return GigaByteSymbol;
if (Math.Abs(MegaBytes) >= 1)
return MegaByteSymbol;
if (Math.Abs(KiloBytes) >= 1)
return KiloByteSymbol;
return ByteSymbol;
}
}
public double LargestWholeNumberValue
{
get
{
// Absolute value is used to deal with negative values
if (Math.Abs(PetaBytes) >= 1)
return PetaBytes;
if (Math.Abs(TeraBytes) >= 1)
return TeraBytes;
if (Math.Abs(GigaBytes) >= 1)
return GigaBytes;
if (Math.Abs(MegaBytes) >= 1)
return MegaBytes;
if (Math.Abs(KiloBytes) >= 1)
return KiloBytes;
return Bytes;
}
}
public FileSize(double bytes)
{
Bytes = bytes;
}
public override string ToString() => $"{LargestWholeNumberValue:0.##} {LargestWholeNumberSymbol}";
}
}
@@ -12,15 +12,15 @@ namespace DiscordChatExporter.Core.Models
public string IconHash { get; }
public string IconUrl => IconHash.IsNotBlank()
? $"https://cdn.discordapp.com/icons/{Id}/{IconHash}.png"
: "https://cdn.discordapp.com/embed/avatars/0.png";
public string IconUrl { get; }
public Guild(string id, string name, string iconHash)
{
Id = id;
Name = name;
IconHash = iconHash;
IconUrl = GetIconUrl(id, iconHash);
}
public override string ToString() => Name;
@@ -28,6 +28,13 @@ namespace DiscordChatExporter.Core.Models
public partial class Guild
{
public static string GetIconUrl(string id, string iconHash)
{
return !iconHash.IsNullOrWhiteSpace()
? $"https://cdn.discordapp.com/icons/{id}/{iconHash}.png"
: "https://cdn.discordapp.com/embed/avatars/0.png";
}
public static Guild DirectMessages { get; } = new Guild("@me", "Direct Messages", null);
}
}
@@ -15,9 +15,9 @@ namespace DiscordChatExporter.Core.Models
public User Author { get; }
public DateTime Timestamp { get; }
public DateTimeOffset Timestamp { get; }
public DateTime? EditedTimestamp { get; }
public DateTimeOffset? EditedTimestamp { get; }
public string Content { get; }
@@ -29,8 +29,8 @@ namespace DiscordChatExporter.Core.Models
public IReadOnlyList<User> MentionedUsers { get; }
public Message(string id, string channelId, MessageType type, User author, DateTime timestamp,
DateTime? editedTimestamp, string content, IReadOnlyList<Attachment> attachments,
public Message(string id, string channelId, MessageType type, User author, DateTimeOffset timestamp,
DateTimeOffset? editedTimestamp, string content, IReadOnlyList<Attachment> attachments,
IReadOnlyList<Embed> embeds, IReadOnlyList<Reaction> reactions, IReadOnlyList<User> mentionedUsers)
{
Id = id;
@@ -19,7 +19,6 @@
public partial class Role
{
public static Role CreateDeletedRole(string id) =>
new Role(id, "deleted-role");
public static Role CreateDeletedRole(string id) => new Role(id, "deleted-role");
}
}
+59
View File
@@ -0,0 +1,59 @@
using System;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Models
{
// https://discordapp.com/developers/docs/topics/permissions#role-object
public partial class User
{
public string Id { get; }
public int Discriminator { get; }
public string Name { get; }
public string FullName { get; }
public string AvatarHash { get; }
public string AvatarUrl { get; }
public User(string id, int discriminator, string name, string avatarHash)
{
Id = id;
Discriminator = discriminator;
Name = name;
AvatarHash = avatarHash;
FullName = GetFullName(name, discriminator);
AvatarUrl = GetAvatarUrl(id, discriminator, avatarHash);
}
public override string ToString() => FullName;
}
public partial class User
{
public static string GetFullName(string name, int discriminator) => $"{name}#{discriminator:0000}";
public static string GetAvatarUrl(string id, int discriminator, string avatarHash)
{
// Custom avatar
if (!avatarHash.IsNullOrWhiteSpace())
{
// Animated
if (avatarHash.StartsWith("a_", StringComparison.Ordinal))
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.gif";
// Non-animated
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.png";
}
// Default avatar
return $"https://cdn.discordapp.com/embed/avatars/{discriminator % 5}.png";
}
public static User CreateUnknownUser(string id) => new User(id, 0, "Unknown", null);
}
}
@@ -0,0 +1,113 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Markdown.Nodes;
using DiscordChatExporter.Core.Models;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Rendering
{
public class CsvChatLogRenderer : IChatLogRenderer
{
private readonly ChatLog _chatLog;
private readonly string _dateFormat;
public CsvChatLogRenderer(ChatLog chatLog, string dateFormat)
{
_chatLog = chatLog;
_dateFormat = dateFormat;
}
private string FormatDate(DateTimeOffset date) =>
date.ToLocalTime().ToString(_dateFormat, CultureInfo.InvariantCulture);
private string FormatMarkdown(Node node)
{
// Formatted node
if (node is FormattedNode formattedNode)
{
// Recursively get inner text
var innerText = FormatMarkdown(formattedNode.Children);
return $"{formattedNode.Token}{innerText}{formattedNode.Token}";
}
// Non-meta mention node
if (node is MentionNode mentionNode && mentionNode.Type != MentionType.Meta)
{
// User mention node
if (mentionNode.Type == MentionType.User)
{
var user = _chatLog.Mentionables.GetUser(mentionNode.Id);
return $"@{user.Name}";
}
// Channel mention node
if (mentionNode.Type == MentionType.Channel)
{
var channel = _chatLog.Mentionables.GetChannel(mentionNode.Id);
return $"#{channel.Name}";
}
// Role mention node
if (mentionNode.Type == MentionType.Role)
{
var role = _chatLog.Mentionables.GetRole(mentionNode.Id);
return $"@{role.Name}";
}
}
// Custom emoji node
if (node is EmojiNode emojiNode && emojiNode.IsCustomEmoji)
{
return $":{emojiNode.Name}:";
}
// All other nodes - simply return source
return node.Source;
}
private string FormatMarkdown(IEnumerable<Node> nodes) => nodes.Select(FormatMarkdown).JoinToString("");
private string FormatMarkdown(string markdown) => FormatMarkdown(MarkdownParser.Parse(markdown));
private async Task RenderFieldAsync(TextWriter writer, string value)
{
var encodedValue = value.Replace("\"", "\"\"");
await writer.WriteAsync($"\"{encodedValue}\";");
}
private async Task RenderMessageAsync(TextWriter writer, Message message)
{
// Author
await RenderFieldAsync(writer, message.Author.FullName);
// Timestamp
await RenderFieldAsync(writer, FormatDate(message.Timestamp));
// Content
await RenderFieldAsync(writer, FormatMarkdown(message.Content));
// Attachments
var formattedAttachments = message.Attachments.Select(a => a.Url).JoinToString(",");
await RenderFieldAsync(writer, formattedAttachments);
// Line break
await writer.WriteLineAsync();
}
public async Task RenderAsync(TextWriter writer)
{
// Headers
await writer.WriteLineAsync("Author;Date;Content;Attachments;");
// Log
foreach (var message in _chatLog.Messages)
await RenderMessageAsync(writer, message);
}
}
}
@@ -0,0 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net46;netstandard2.0</TargetFrameworks>
</PropertyGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\HtmlDark.css" />
<EmbeddedResource Include="Resources\HtmlDark.html" />
<EmbeddedResource Include="Resources\HtmlLight.css" />
<EmbeddedResource Include="Resources\HtmlLight.html" />
<EmbeddedResource Include="Resources\HtmlShared.css" />
<EmbeddedResource Include="Resources\HtmlShared.html" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Scriban" Version="2.0.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core.Markdown\DiscordChatExporter.Core.Markdown.csproj" />
<ProjectReference Include="..\DiscordChatExporter.Core.Models\DiscordChatExporter.Core.Models.csproj" />
</ItemGroup>
</Project>
@@ -1,20 +1,20 @@
using System;
using DiscordChatExporter.Core.Models;
using System;
using System.Collections.Generic;
using DiscordChatExporter.Core.Models;
namespace DiscordChatExporter.Core.Services
namespace DiscordChatExporter.Core.Rendering
{
public partial class ExportService
public partial class HtmlChatLogRenderer
{
private class MessageGroup
{
public User Author { get; }
public DateTime Timestamp { get; }
public DateTimeOffset Timestamp { get; }
public IReadOnlyList<Message> Messages { get; }
public MessageGroup(User author, DateTime timestamp, IReadOnlyList<Message> messages)
public MessageGroup(User author, DateTimeOffset timestamp, IReadOnlyList<Message> messages)
{
Author = author;
Timestamp = timestamp;
@@ -0,0 +1,27 @@
using Scriban.Parsing;
using Scriban.Runtime;
using Scriban;
using System.Reflection;
using System.Threading.Tasks;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Rendering
{
public partial class HtmlChatLogRenderer
{
private class TemplateLoader : ITemplateLoader
{
private const string ResourceRootNamespace = "DiscordChatExporter.Core.Rendering.Resources";
public string Load(string templatePath) =>
Assembly.GetExecutingAssembly().GetManifestResourceString($"{ResourceRootNamespace}.{templatePath}");
public string GetPath(TemplateContext context, SourceSpan callerSpan, string templateName) => templateName;
public string Load(TemplateContext context, SourceSpan callerSpan, string templatePath) => Load(templatePath);
public ValueTask<string> LoadAsync(TemplateContext context, SourceSpan callerSpan, string templatePath) =>
new ValueTask<string>(Load(templatePath));
}
}
}
@@ -0,0 +1,200 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Markdown.Nodes;
using DiscordChatExporter.Core.Models;
using Scriban;
using Scriban.Runtime;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Rendering
{
public partial class HtmlChatLogRenderer : IChatLogRenderer
{
private readonly ChatLog _chatLog;
private readonly string _themeName;
private readonly string _dateFormat;
public HtmlChatLogRenderer(ChatLog chatLog, string themeName, string dateFormat)
{
_chatLog = chatLog;
_themeName = themeName;
_dateFormat = dateFormat;
}
private string HtmlEncode(string s) => WebUtility.HtmlEncode(s);
private string FormatDate(DateTimeOffset date) =>
date.ToLocalTime().ToString(_dateFormat, CultureInfo.InvariantCulture);
private IEnumerable<MessageGroup> GroupMessages(IEnumerable<Message> messages) =>
messages.GroupContiguous((buffer, message) =>
{
// Break group if the author changed
if (buffer.Last().Author.Id != message.Author.Id)
return false;
// Break group if last message was more than 7 minutes ago
if ((message.Timestamp - buffer.Last().Timestamp).TotalMinutes > 7)
return false;
return true;
}).Select(g => new MessageGroup(g.First().Author, g.First().Timestamp, g));
private string FormatMarkdown(Node node, bool isJumbo)
{
// Text node
if (node is TextNode textNode)
{
// Return HTML-encoded text
return HtmlEncode(textNode.Text);
}
// Formatted node
if (node is FormattedNode formattedNode)
{
// Recursively get inner html
var innerHtml = FormatMarkdown(formattedNode.Children, false);
// Bold
if (formattedNode.Formatting == TextFormatting.Bold)
return $"<strong>{innerHtml}</strong>";
// Italic
if (formattedNode.Formatting == TextFormatting.Italic)
return $"<em>{innerHtml}</em>";
// Underline
if (formattedNode.Formatting == TextFormatting.Underline)
return $"<u>{innerHtml}</u>";
// Strikethrough
if (formattedNode.Formatting == TextFormatting.Strikethrough)
return $"<s>{innerHtml}</s>";
// Spoiler
if (formattedNode.Formatting == TextFormatting.Spoiler)
return $"<span class=\"spoiler\">{innerHtml}</span>";
}
// Inline code block node
if (node is InlineCodeBlockNode inlineCodeBlockNode)
{
return $"<span class=\"pre pre--inline\">{HtmlEncode(inlineCodeBlockNode.Code)}</span>";
}
// Multi-line code block node
if (node is MultilineCodeBlockNode multilineCodeBlockNode)
{
// Set language class for syntax highlighting
var languageCssClass = !multilineCodeBlockNode.Language.IsNullOrWhiteSpace()
? "language-" + multilineCodeBlockNode.Language
: null;
return $"<div class=\"pre pre--multiline {languageCssClass}\">{HtmlEncode(multilineCodeBlockNode.Code)}</div>";
}
// Mention node
if (node is MentionNode mentionNode)
{
// Meta mention node
if (mentionNode.Type == MentionType.Meta)
{
return $"<span class=\"mention\">@{HtmlEncode(mentionNode.Id)}</span>";
}
// User mention node
if (mentionNode.Type == MentionType.User)
{
var user = _chatLog.Mentionables.GetUser(mentionNode.Id);
return $"<span class=\"mention\" title=\"{HtmlEncode(user.FullName)}\">@{HtmlEncode(user.Name)}</span>";
}
// Channel mention node
if (mentionNode.Type == MentionType.Channel)
{
var channel = _chatLog.Mentionables.GetChannel(mentionNode.Id);
return $"<span class=\"mention\">#{HtmlEncode(channel.Name)}</span>";
}
// Role mention node
if (mentionNode.Type == MentionType.Role)
{
var role = _chatLog.Mentionables.GetRole(mentionNode.Id);
return $"<span class=\"mention\">@{HtmlEncode(role.Name)}</span>";
}
}
// Emoji node
if (node is EmojiNode emojiNode)
{
// Get emoji image URL
var emojiImageUrl = Emoji.GetImageUrl(emojiNode.Id, emojiNode.Name, emojiNode.IsAnimated);
// Make emoji large if it's jumbo
var jumboableCssClass = isJumbo ? "emoji--large" : null;
return $"<img class=\"emoji {jumboableCssClass}\" alt=\"{emojiNode.Name}\" title=\"{emojiNode.Name}\" src=\"{emojiImageUrl}\" />";
}
// Link node
if (node is LinkNode linkNode)
{
return $"<a href=\"{Uri.EscapeUriString(linkNode.Url)}\">{HtmlEncode(linkNode.Title)}</a>";
}
// All other nodes - simply return source
return node.Source;
}
private string FormatMarkdown(IReadOnlyList<Node> nodes, bool isTopLevel)
{
// Emojis are jumbo if all top-level nodes are emoji nodes, disregarding whitespace
var isJumbo = isTopLevel && nodes.Where(n => !n.Source.IsNullOrWhiteSpace()).All(n => n is EmojiNode);
return nodes.Select(n => FormatMarkdown(n, isJumbo)).JoinToString("");
}
private string FormatMarkdown(string markdown) => FormatMarkdown(MarkdownParser.Parse(markdown), true);
public async Task RenderAsync(TextWriter writer)
{
// Create template loader
var loader = new TemplateLoader();
// Get template
var templateCode = loader.Load($"Html{_themeName}.html");
var template = Template.Parse(templateCode);
// Create template context
var context = new TemplateContext
{
TemplateLoader = loader,
MemberRenamer = m => m.Name,
MemberFilter = m => true,
LoopLimit = int.MaxValue,
StrictVariables = true
};
// Create template model
var model = new ScriptObject();
model.SetValue("Model", _chatLog, true);
model.Import(nameof(GroupMessages), new Func<IEnumerable<Message>, IEnumerable<MessageGroup>>(GroupMessages));
model.Import(nameof(FormatDate), new Func<DateTimeOffset, string>(FormatDate));
model.Import(nameof(FormatMarkdown), new Func<string, string>(FormatMarkdown));
context.PushGlobal(model);
// Configure output
context.PushOutput(new TextWriterOutput(writer));
// HACK: Render output in a separate thread
// (even though Scriban has async API, it still makes a lot of blocking CPU-bound calls)
await Task.Run(async () => await context.EvaluateAsync(template.Page));
}
}
}
@@ -0,0 +1,10 @@
using System.IO;
using System.Threading.Tasks;
namespace DiscordChatExporter.Core.Rendering
{
public interface IChatLogRenderer
{
Task RenderAsync(TextWriter writer);
}
}
@@ -0,0 +1,129 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Markdown.Nodes;
using DiscordChatExporter.Core.Models;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Rendering
{
public class PlainTextChatLogRenderer : IChatLogRenderer
{
private readonly ChatLog _chatLog;
private readonly string _dateFormat;
public PlainTextChatLogRenderer(ChatLog chatLog, string dateFormat)
{
_chatLog = chatLog;
_dateFormat = dateFormat;
}
private string FormatDate(DateTimeOffset date) =>
date.ToLocalTime().ToString(_dateFormat, CultureInfo.InvariantCulture);
private string FormatDateRange(DateTimeOffset? after, DateTimeOffset? before)
{
// Both 'after' and 'before'
if (after != null && before != null)
return $"{FormatDate(after.Value)} to {FormatDate(before.Value)}";
// Just 'after'
if (after != null)
return $"after {FormatDate(after.Value)}";
// Just 'before'
if (before != null)
return $"before {FormatDate(before.Value)}";
// Neither
return null;
}
private string FormatMarkdown(Node node)
{
// Formatted node
if (node is FormattedNode formattedNode)
{
// Recursively get inner text
var innerText = FormatMarkdown(formattedNode.Children);
return $"{formattedNode.Token}{innerText}{formattedNode.Token}";
}
// Non-meta mention node
if (node is MentionNode mentionNode && mentionNode.Type != MentionType.Meta)
{
// User mention node
if (mentionNode.Type == MentionType.User)
{
var user = _chatLog.Mentionables.GetUser(mentionNode.Id);
return $"@{user.Name}";
}
// Channel mention node
if (mentionNode.Type == MentionType.Channel)
{
var channel = _chatLog.Mentionables.GetChannel(mentionNode.Id);
return $"#{channel.Name}";
}
// Role mention node
if (mentionNode.Type == MentionType.Role)
{
var role = _chatLog.Mentionables.GetRole(mentionNode.Id);
return $"@{role.Name}";
}
}
// Custom emoji node
if (node is EmojiNode emojiNode && emojiNode.IsCustomEmoji)
{
return $":{emojiNode.Name}:";
}
// All other nodes - simply return source
return node.Source;
}
private string FormatMarkdown(IEnumerable<Node> nodes) => nodes.Select(FormatMarkdown).JoinToString("");
private string FormatMarkdown(string markdown) => FormatMarkdown(MarkdownParser.Parse(markdown));
private async Task RenderMessageAsync(TextWriter writer, Message message)
{
// Timestamp and author
await writer.WriteLineAsync($"[{FormatDate(message.Timestamp)}] {message.Author.FullName}");
// Content
await writer.WriteLineAsync(FormatMarkdown(message.Content));
// Attachments
foreach (var attachment in message.Attachments)
await writer.WriteLineAsync(attachment.Url);
}
public async Task RenderAsync(TextWriter writer)
{
// Metadata
await writer.WriteLineAsync('='.Repeat(62));
await writer.WriteLineAsync($"Guild: {_chatLog.Guild.Name}");
await writer.WriteLineAsync($"Channel: {_chatLog.Channel.Name}");
await writer.WriteLineAsync($"Topic: {_chatLog.Channel.Topic}");
await writer.WriteLineAsync($"Messages: {_chatLog.Messages.Count:N0}");
await writer.WriteLineAsync($"Range: {FormatDateRange(_chatLog.After, _chatLog.Before)}");
await writer.WriteLineAsync('='.Repeat(62));
await writer.WriteLineAsync();
// Log
foreach (var message in _chatLog.Messages)
{
await RenderMessageAsync(writer, message);
await writer.WriteLineAsync();
}
}
}
}
@@ -14,12 +14,12 @@ a {
}
.pre {
background-color: #2f3136;
background-color: #2f3136 !important;
}
.pre--multiline {
border-color: #282b30;
color: #839496;
border-color: #282b30 !important;
color: #839496 !important;
}
.mention {
@@ -0,0 +1,3 @@
{{~ ThemeStyleSheet = include "HtmlDark.css" ~}}
{{~ HighlightJsStyleName = "solarized-dark" ~}}
{{~ include "HtmlShared.html" ~}}
@@ -14,12 +14,12 @@ a {
}
.pre {
background-color: #f9f9f9;
background-color: #f9f9f9 !important;
}
.pre--multiline {
border-color: #f3f3f3;
color: #657b83;
border-color: #f3f3f3 !important;
color: #657b83 !important;
}
.mention {
@@ -0,0 +1,3 @@
{{~ ThemeStyleSheet = include "HtmlLight.css" ~}}
{{~ HighlightJsStyleName = "solarized-light" ~}}
{{~ include "HtmlShared.html" ~}}
@@ -110,6 +110,7 @@ img {
.chatlog {
max-width: 100%;
margin-bottom: 24px;
}
.chatlog__message-group {
@@ -2,15 +2,29 @@
<html lang="en">
<head>
{{~ # Metadata ~}}
<title>{{ Model.Guild.Name | html.escape }} - {{ Model.Channel.Name | html.escape }}</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width" />
{{~ # Styles ~}}
<style>
{{ include "HtmlShared.Main.css" }}
{{ include "HtmlShared.css" }}
</style>
<style>
{{ ThemeStyleSheet }}
</style>
{{~ # Syntax highlighting ~}}
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/styles/{{HighlightJsStyleName}}.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/highlight.min.js"></script>
<script>
document.addEventListener('DOMContentLoaded', () => {
document.querySelectorAll('.pre--multiline').forEach((block) => {
hljs.highlightBlock(block);
});
});
</script>
</head>
<body>
@@ -27,16 +41,16 @@
<div class="info__channel-topic">{{ Model.Channel.Topic | html.escape }}</div>
{{~ end ~}}
<div class="info__channel-message-count">{{ Model.Messages | array.size | Format "N0" }} messages</div>
<div class="info__channel-message-count">{{ Model.Messages | array.size | object.format "N0" }} messages</div>
{{~ if Model.From || Model.To ~}}
{{~ if Model.After || Model.Before ~}}
<div class="info__channel-date-range">
{{~ if Model.From && Model.To ~}}
Between {{ Model.From | FormatDate | html.escape }} and {{ Model.To | FormatDate | html.escape }}
{{~ else if Model.From ~}}
After {{ Model.From | FormatDate | html.escape }}
{{~ else if Model.To ~}}
Before {{ Model.To | FormatDate | html.escape }}
{{~ if Model.After && Model.Before ~}}
Between {{ Model.After | FormatDate | html.escape }} and {{ Model.Before | FormatDate | html.escape }}
{{~ else if Model.After ~}}
After {{ Model.After | FormatDate | html.escape }}
{{~ else if Model.Before ~}}
Before {{ Model.Before | FormatDate | html.escape }}
{{~ end ~}}
</div>
{{~ end ~}}
@@ -1,9 +1,8 @@
using System;
using System.Drawing;
using System.Linq;
using ByteSizeLib;
using DiscordChatExporter.Core.Internal;
using DiscordChatExporter.Core.Models;
using DiscordChatExporter.Core.Services.Internal;
using Newtonsoft.Json.Linq;
using Tyrrrz.Extensions;
@@ -42,14 +41,14 @@ namespace DiscordChatExporter.Core.Services
var guildId = json["guild_id"]?.Value<string>();
// If the guild ID is blank, it's direct messages
if (guildId.IsBlank())
if (guildId.IsNullOrWhiteSpace())
guildId = Guild.DirectMessages.Id;
// Try to extract name
var name = json["name"]?.Value<string>();
// If the name is blank, it's direct messages
if (name.IsBlank())
if (name.IsNullOrWhiteSpace())
name = json["recipients"].Select(ParseUser).Select(u => u.Name).JoinToString(", ");
return new Channel(id, parentId, guildId, name, topic, type);
@@ -72,7 +71,7 @@ namespace DiscordChatExporter.Core.Services
var fileName = json["filename"].Value<string>();
var fileSizeBytes = json["size"].Value<long>();
var fileSize = ByteSize.FromBytes(fileSizeBytes);
var fileSize = new FileSize(fileSizeBytes);
return new Attachment(id, width, height, url, fileName, fileSize);
}
@@ -118,7 +117,7 @@ namespace DiscordChatExporter.Core.Services
var title = json["title"]?.Value<string>();
var description = json["description"]?.Value<string>();
var url = json["url"]?.Value<string>();
var timestamp = json["timestamp"]?.Value<DateTime>();
var timestamp = json["timestamp"]?.Value<DateTimeOffset>();
// Get color
var color = json["color"] != null
@@ -165,8 +164,8 @@ namespace DiscordChatExporter.Core.Services
// Get basic data
var id = json["id"].Value<string>();
var channelId = json["channel_id"].Value<string>();
var timestamp = json["timestamp"].Value<DateTime>();
var editedTimestamp = json["edited_timestamp"]?.Value<DateTime?>();
var timestamp = json["timestamp"].Value<DateTimeOffset>();
var editedTimestamp = json["edited_timestamp"]?.Value<DateTimeOffset?>();
var content = json["content"].Value<string>();
var type = (MessageType) json["type"].Value<int>();
@@ -1,15 +1,16 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Models;
using Newtonsoft.Json.Linq;
using DiscordChatExporter.Core.Internal;
using DiscordChatExporter.Core.Services.Exceptions;
using DiscordChatExporter.Core.Services.Internal;
using Failsafe;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Services
@@ -23,7 +24,7 @@ namespace DiscordChatExporter.Core.Services
{
// Create retry policy
var retry = Retry.Create()
.Catch<HttpErrorStatusCodeException>(false, e => e.StatusCode == HttpStatusCode.ServiceUnavailable)
.Catch<HttpErrorStatusCodeException>(false, e => (int) e.StatusCode >= 500)
.Catch<HttpErrorStatusCodeException>(false, e => (int) e.StatusCode == 429)
.WithMaxTryCount(10)
.WithDelay(TimeSpan.FromSeconds(0.4));
@@ -41,13 +42,13 @@ namespace DiscordChatExporter.Core.Services
: new AuthenticationHeaderValue(token.Value);
// Add parameters
foreach (var parameter in parameters.ExceptBlank())
foreach (var parameter in parameters)
{
var key = parameter.SubstringUntil("=");
var value = parameter.SubstringAfter("=");
// Skip empty values
if (value.IsBlank())
if (value.IsNullOrWhiteSpace())
continue;
request.RequestUri = request.RequestUri.SetQueryParameter(key, value);
@@ -65,7 +66,12 @@ namespace DiscordChatExporter.Core.Services
var raw = await response.Content.ReadAsStringAsync();
// Parse
return JToken.Parse(raw);
using (var reader = new JsonTextReader(new StringReader(raw)))
{
reader.DateParseHandling = DateParseHandling.DateTimeOffset;
return JToken.Load(reader);
}
}
}
});
@@ -124,24 +130,24 @@ namespace DiscordChatExporter.Core.Services
}
public async Task<IReadOnlyList<Message>> GetChannelMessagesAsync(AuthToken token, string channelId,
DateTime? from = null, DateTime? to = null, IProgress<double> progress = null)
DateTimeOffset? after = null, DateTimeOffset? before = null, IProgress<double> progress = null)
{
var result = new List<Message>();
// Get the last message
var response = await GetApiResponseAsync(token, "channels", $"{channelId}/messages",
"limit=1", $"before={to?.ToSnowflake()}");
"limit=1", $"before={before?.ToSnowflake()}");
var lastMessage = response.Select(ParseMessage).FirstOrDefault();
// If the last message doesn't exist or it's outside of range - return
if (lastMessage == null || lastMessage.Timestamp < from)
if (lastMessage == null || lastMessage.Timestamp < after)
{
progress?.Report(1);
return result;
}
// Get other messages
var offsetId = from?.ToSnowflake() ?? "0";
var offsetId = after?.ToSnowflake() ?? "0";
while (true)
{
// Get message batch
@@ -216,19 +222,19 @@ namespace DiscordChatExporter.Core.Services
}
public async Task<ChatLog> GetChatLogAsync(AuthToken token, Guild guild, Channel channel,
DateTime? from = null, DateTime? to = null, IProgress<double> progress = null)
DateTimeOffset? after = null, DateTimeOffset? before = null, IProgress<double> progress = null)
{
// Get messages
var messages = await GetChannelMessagesAsync(token, channel.Id, from, to, progress);
var messages = await GetChannelMessagesAsync(token, channel.Id, after, before, progress);
// Get mentionables
var mentionables = await GetMentionablesAsync(token, guild.Id, messages);
return new ChatLog(guild, channel, from, to, messages, mentionables);
return new ChatLog(guild, channel, after, before, messages, mentionables);
}
public async Task<ChatLog> GetChatLogAsync(AuthToken token, Channel channel,
DateTime? from = null, DateTime? to = null, IProgress<double> progress = null)
DateTimeOffset? after = null, DateTimeOffset? before = null, IProgress<double> progress = null)
{
// Get guild
var guild = channel.GuildId == Guild.DirectMessages.Id
@@ -236,17 +242,17 @@ namespace DiscordChatExporter.Core.Services
: await GetGuildAsync(token, channel.GuildId);
// Get the chat log
return await GetChatLogAsync(token, guild, channel, from, to, progress);
return await GetChatLogAsync(token, guild, channel, after, before, progress);
}
public async Task<ChatLog> GetChatLogAsync(AuthToken token, string channelId,
DateTime? from = null, DateTime? to = null, IProgress<double> progress = null)
DateTimeOffset? after = null, DateTimeOffset? before = null, IProgress<double> progress = null)
{
// Get channel
var channel = await GetChannelAsync(token, channelId);
// Get the chat log
return await GetChatLogAsync(token, channel, from, to, progress);
return await GetChatLogAsync(token, channel, after, before, progress);
}
public void Dispose()
@@ -0,0 +1,20 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net46;netstandard2.0</TargetFrameworks>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Failsafe" Version="1.1.0" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.1" />
<PackageReference Include="Onova" Version="2.4.2" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.1" />
<PackageReference Include="Tyrrrz.Settings" Version="1.3.4" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core.Models\DiscordChatExporter.Core.Models.csproj" />
<ProjectReference Include="..\DiscordChatExporter.Core.Rendering\DiscordChatExporter.Core.Rendering.csproj" />
</ItemGroup>
</Project>
@@ -1,7 +1,7 @@
using System;
using System.Net;
namespace DiscordChatExporter.Core.Exceptions
namespace DiscordChatExporter.Core.Services.Exceptions
{
public class HttpErrorStatusCodeException : Exception
{
@@ -0,0 +1,89 @@
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Models;
using DiscordChatExporter.Core.Rendering;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Services
{
public class ExportService
{
private readonly SettingsService _settingsService;
public ExportService(SettingsService settingsService)
{
_settingsService = settingsService;
}
private IChatLogRenderer CreateRenderer(ChatLog chatLog, ExportFormat format)
{
if (format == ExportFormat.PlainText)
return new PlainTextChatLogRenderer(chatLog, _settingsService.DateFormat);
if (format == ExportFormat.HtmlDark)
return new HtmlChatLogRenderer(chatLog, "Dark", _settingsService.DateFormat);
if (format == ExportFormat.HtmlLight)
return new HtmlChatLogRenderer(chatLog, "Light", _settingsService.DateFormat);
if (format == ExportFormat.Csv)
return new CsvChatLogRenderer(chatLog, _settingsService.DateFormat);
throw new ArgumentOutOfRangeException(nameof(format), $"Unknown format [{format}].");
}
private async Task ExportChatLogAsync(ChatLog chatLog, string filePath, ExportFormat format)
{
// Create output directory
var dirPath = Path.GetDirectoryName(filePath);
if (!dirPath.IsNullOrWhiteSpace())
Directory.CreateDirectory(dirPath);
// Render chat log to output file
using (var writer = File.CreateText(filePath))
await CreateRenderer(chatLog, format).RenderAsync(writer);
}
public async Task ExportChatLogAsync(ChatLog chatLog, string filePath, ExportFormat format, int? partitionLimit)
{
// If partitioning is disabled or there are fewer messages in chat log than the limit - process it without partitioning
if (partitionLimit == null || partitionLimit <= 0 || chatLog.Messages.Count <= partitionLimit)
{
await ExportChatLogAsync(chatLog, filePath, format);
}
// Otherwise split into partitions and export separately
else
{
// Create partitions by grouping up to X contiguous messages into separate chat logs
var partitions = chatLog.Messages.GroupContiguous(g => g.Count < partitionLimit.Value)
.Select(g => new ChatLog(chatLog.Guild, chatLog.Channel, chatLog.After, chatLog.Before, g, chatLog.Mentionables))
.ToArray();
// Split file path into components
var dirPath = Path.GetDirectoryName(filePath);
var fileNameWithoutExt = Path.GetFileNameWithoutExtension(filePath);
var fileExt = Path.GetExtension(filePath);
// Export each partition separately
var partitionNumber = 1;
foreach (var partition in partitions)
{
// Compose new file name
var partitionFilePath = $"{fileNameWithoutExt} [{partitionNumber} of {partitions.Length}]{fileExt}";
// Compose full file path
if (!dirPath.IsNullOrWhiteSpace())
partitionFilePath = Path.Combine(dirPath, partitionFilePath);
// Export
await ExportChatLogAsync(partition, partitionFilePath, format);
// Increment partition number
partitionNumber++;
}
}
}
}
}
@@ -5,17 +5,17 @@ using System.Text;
using DiscordChatExporter.Core.Models;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Helpers
namespace DiscordChatExporter.Core.Services.Helpers
{
public static class ExportHelper
{
public static bool IsDirectoryPath(string path)
=> path.Last() == Path.DirectorySeparatorChar ||
path.Last() == Path.AltDirectorySeparatorChar ||
Path.GetExtension(path).IsBlank();
public static bool IsDirectoryPath(string path) =>
path.Last() == Path.DirectorySeparatorChar ||
path.Last() == Path.AltDirectorySeparatorChar ||
(Path.GetExtension(path).IsNullOrWhiteSpace() && !File.Exists(path));
public static string GetDefaultExportFileName(ExportFormat format, Guild guild, Channel channel,
DateTime? from = null, DateTime? to = null)
DateTimeOffset? after = null, DateTimeOffset? before = null)
{
var result = new StringBuilder();
@@ -23,24 +23,24 @@ namespace DiscordChatExporter.Core.Helpers
result.Append($"{guild.Name} - {channel.Name} [{channel.Id}]");
// Append date range
if (from != null || to != null)
if (after != null || before != null)
{
result.Append(" (");
// Both 'from' and 'to' are set
if (from != null && to != null)
// Both 'after' and 'before' are set
if (after != null && before != null)
{
result.Append($"{from:yyyy-MM-dd} to {to:yyyy-MM-dd}");
result.Append($"{after:yyyy-MM-dd} to {before:yyyy-MM-dd}");
}
// Only 'from' is set
else if (from != null)
// Only 'after' is set
else if (after != null)
{
result.Append($"after {from:yyyy-MM-dd}");
result.Append($"after {after:yyyy-MM-dd}");
}
// Only 'to' is set
// Only 'before' is set
else
{
result.Append($"before {to:yyyy-MM-dd}");
result.Append($"before {before:yyyy-MM-dd}");
}
result.Append(")");
@@ -1,21 +1,18 @@
using System;
using System.Drawing;
using System.Net;
namespace DiscordChatExporter.Core.Internal
namespace DiscordChatExporter.Core.Services.Internal
{
internal static class Extensions
{
public static string ToSnowflake(this DateTime dateTime)
public static string ToSnowflake(this DateTimeOffset date)
{
const long epoch = 62135596800000;
var unixTime = dateTime.ToUniversalTime().Ticks / TimeSpan.TicksPerMillisecond - epoch;
var unixTime = date.ToUniversalTime().Ticks / TimeSpan.TicksPerMillisecond - epoch;
var value = ((ulong) unixTime - 1420070400000UL) << 22;
return value.ToString();
}
public static Color ResetAlpha(this Color color) => Color.FromArgb(1, color);
public static string HtmlEncode(this string value) => WebUtility.HtmlEncode(value);
}
}
@@ -8,7 +8,6 @@ namespace DiscordChatExporter.Core.Services
public bool IsAutoUpdateEnabled { get; set; } = true;
public string DateFormat { get; set; } = "dd-MMM-yy hh:mm tt";
public int MessageGroupLimit { get; set; } = 20;
public AuthToken LastToken { get; set; }
public ExportFormat LastExportFormat { get; set; } = ExportFormat.HtmlDark;
@@ -0,0 +1,79 @@
using System;
using System.Threading.Tasks;
using Onova;
using Onova.Exceptions;
using Onova.Services;
namespace DiscordChatExporter.Core.Services
{
public class UpdateService : IDisposable
{
private readonly SettingsService _settingsService;
private readonly IUpdateManager _updateManager = new UpdateManager(
new GithubPackageResolver("Tyrrrz", "DiscordChatExporter", "DiscordChatExporter.zip"),
new ZipPackageExtractor());
private Version _updateVersion;
private bool _updaterLaunched;
public UpdateService(SettingsService settingsService)
{
_settingsService = settingsService;
}
public async Task<Version> CheckPrepareUpdateAsync()
{
try
{
// If auto-update is disabled - don't check for updates
if (!_settingsService.IsAutoUpdateEnabled)
return null;
// Check for updates
var check = await _updateManager.CheckForUpdatesAsync();
if (!check.CanUpdate)
return null;
// Prepare the update
await _updateManager.PrepareUpdateAsync(check.LastVersion);
return _updateVersion = check.LastVersion;
}
catch (UpdaterAlreadyLaunchedException)
{
return null;
}
catch (LockFileNotAcquiredException)
{
return null;
}
}
public void FinalizeUpdate(bool needRestart)
{
try
{
// Check if an update is pending
if (_updateVersion == null)
return;
// Check if the updater has already been launched
if (_updaterLaunched)
return;
// Launch the updater
_updateManager.LaunchUpdater(_updateVersion, needRestart);
_updaterLaunched = true;
}
catch (UpdaterAlreadyLaunchedException)
{
}
catch (LockFileNotAcquiredException)
{
}
}
public void Dispose() => _updateManager.Dispose();
}
}
@@ -1,36 +0,0 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net461</TargetFramework>
</PropertyGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\ExportTemplates\PlainText\Template.txt" />
<EmbeddedResource Include="Resources\ExportTemplates\HtmlDark\Template.html" />
<EmbeddedResource Include="Resources\ExportTemplates\HtmlLight\Template.html" />
<EmbeddedResource Include="Resources\ExportTemplates\HtmlShared\Main.html" />
<EmbeddedResource Include="Resources\ExportTemplates\HtmlShared\Main.css" />
<EmbeddedResource Include="Resources\ExportTemplates\HtmlDark\Theme.css" />
<EmbeddedResource Include="Resources\ExportTemplates\HtmlLight\Theme.css" />
<EmbeddedResource Include="Resources\ExportTemplates\Csv\Template.csv" />
</ItemGroup>
<ItemGroup>
<Reference Include="mscorlib" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="ByteSize" Version="1.3.0" />
<PackageReference Include="Failsafe" Version="1.1.0" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.1" />
<PackageReference Include="Onova" Version="2.2.0" />
<PackageReference Include="Scriban" Version="1.2.9" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.5.1" />
<PackageReference Include="Tyrrrz.Settings" Version="1.3.3" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core.Markdown\DiscordChatExporter.Core.Markdown.csproj" />
</ItemGroup>
</Project>
-61
View File
@@ -1,61 +0,0 @@
using System;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Models
{
// https://discordapp.com/developers/docs/topics/permissions#role-object
public partial class User
{
public string Id { get; }
public int Discriminator { get; }
public string Name { get; }
public string FullName => $"{Name}#{Discriminator:0000}";
public string DefaultAvatarHash => $"{Discriminator % 5}";
public string AvatarHash { get; }
public bool IsAvatarAnimated =>
AvatarHash.IsNotBlank() && AvatarHash.StartsWith("a_", StringComparison.Ordinal);
public string AvatarUrl
{
get
{
// Custom avatar
if (AvatarHash.IsNotBlank())
{
// Animated
if (IsAvatarAnimated)
return $"https://cdn.discordapp.com/avatars/{Id}/{AvatarHash}.gif";
// Non-animated
return $"https://cdn.discordapp.com/avatars/{Id}/{AvatarHash}.png";
}
// Default avatar
return $"https://cdn.discordapp.com/embed/avatars/{DefaultAvatarHash}.png";
}
}
public User(string id, int discriminator, string name, string avatarHash)
{
Id = id;
Discriminator = discriminator;
Name = name;
AvatarHash = avatarHash;
}
public override string ToString() => FullName;
}
public partial class User
{
public static User CreateUnknownUser(string id) =>
new User(id, 0, "Unknown", null);
}
}
@@ -1,10 +0,0 @@
Author;Date;Content;Attachments;
{{~ for message in Model.Messages -}}
{{- }}"{{ message.Author.FullName }}";
{{- }}"{{ message.Timestamp | FormatDate }}";
{{- }}"{{ message.Content | FormatMarkdown | string.replace "\"" "\"\"" }}";
{{- }}"{{ message.Attachments | array.map "Url" | array.join "," }}";
{{~ end -}}
Can't render this file because it contains an unexpected character in line 3 and column 9.
@@ -1,2 +0,0 @@
{{~ ThemeStyleSheet = include "HtmlDark.Theme.css" ~}}
{{~ include "HtmlShared.Main.html" ~}}
@@ -1,2 +0,0 @@
{{~ ThemeStyleSheet = include "HtmlLight.Theme.css" ~}}
{{~ include "HtmlShared.Main.html" ~}}
@@ -1,21 +0,0 @@
{{~ # Info ~}}
==============================================================
Guild: {{ Model.Guild.Name }}
Channel: {{ Model.Channel.Name }}
Topic: {{ Model.Channel.Topic }}
Messages: {{ Model.Messages | array.size | Format "N0" }}
Range: {{ if Model.From }}{{ Model.From | FormatDate }} {{ end }}{{ if Model.From || Model.To }}->{{ end }}{{ if Model.To }} {{ Model.To | FormatDate }}{{ end }}
==============================================================
{{~ # Log ~}}
{{~ for message in Model.Messages ~}}
{{~ # Author name and timestamp ~}}
{{~ }}[{{ message.Timestamp | FormatDate }}] {{ message.Author.FullName }}
{{~ # Content ~}}
{{~ message.Content | FormatMarkdown }}
{{~ # Attachments ~}}
{{~ for attachment in message.Attachments ~}}
{{~ attachment.Url }}
{{~ end ~}}
{{~ end ~}}
@@ -1,37 +0,0 @@
using System.Reflection;
using DiscordChatExporter.Core.Models;
using Scriban;
using Scriban.Parsing;
using Scriban.Runtime;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Services
{
public partial class ExportService
{
private class TemplateLoader : ITemplateLoader
{
private const string ResourceRootNamespace = "DiscordChatExporter.Core.Resources.ExportTemplates";
public string GetPath(TemplateContext context, SourceSpan callerSpan, string templateName)
{
return $"{ResourceRootNamespace}.{templateName}";
}
public string GetPath(ExportFormat format)
{
return $"{ResourceRootNamespace}.{format}.Template.{format.GetFileExtension()}";
}
public string Load(TemplateContext context, SourceSpan callerSpan, string templatePath)
{
return Assembly.GetExecutingAssembly().GetManifestResourceString(templatePath);
}
public string Load(ExportFormat format)
{
return Assembly.GetExecutingAssembly().GetManifestResourceString(GetPath(format));
}
}
}
}
@@ -1,254 +0,0 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Text;
using DiscordChatExporter.Core.Internal;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Models;
using Scriban.Runtime;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Services
{
public partial class ExportService
{
private class TemplateModel
{
private readonly ExportFormat _format;
private readonly ChatLog _log;
private readonly string _dateFormat;
private readonly int _messageGroupLimit;
public TemplateModel(ExportFormat format, ChatLog log, string dateFormat, int messageGroupLimit)
{
_format = format;
_log = log;
_dateFormat = dateFormat;
_messageGroupLimit = messageGroupLimit;
}
private IEnumerable<MessageGroup> GroupMessages(IEnumerable<Message> messages)
{
// Group adjacent messages by timestamp and author
var buffer = new List<Message>();
foreach (var message in messages)
{
// Get first message of the group
var groupFirstMessage = buffer.FirstOrDefault();
// Group break condition
var breakCondition =
groupFirstMessage != null &&
(
message.Author.Id != groupFirstMessage.Author.Id || // when author changes
(message.Timestamp - groupFirstMessage.Timestamp).TotalHours > 1 || // when difference in timestamps is an hour or more
message.Timestamp.Hour != groupFirstMessage.Timestamp.Hour || // when the timestamp's hour changes
buffer.Count >= _messageGroupLimit // when group is full
);
// If condition is true - flush buffer
if (breakCondition)
{
var group = new MessageGroup(groupFirstMessage.Author, groupFirstMessage.Timestamp, buffer);
// Reset the buffer instead of clearing to avoid mutations on existing references
buffer = new List<Message>();
yield return group;
}
// Add message to buffer
buffer.Add(message);
}
// Add what's remaining in buffer
if (buffer.Any())
{
var groupFirstMessage = buffer.First();
var group = new MessageGroup(groupFirstMessage.Author, groupFirstMessage.Timestamp, buffer);
yield return group;
}
}
private string Format(IFormattable obj, string format) =>
obj.ToString(format, CultureInfo.InvariantCulture);
private string FormatDate(DateTime dateTime) => Format(dateTime, _dateFormat);
private string FormatMarkdownPlainText(IReadOnlyList<Node> nodes)
{
var buffer = new StringBuilder();
foreach (var node in nodes)
{
if (node is FormattedNode formattedNode)
{
var innerText = FormatMarkdownPlainText(formattedNode.Children);
buffer.Append($"{formattedNode.Token}{innerText}{formattedNode.Token}");
}
else if (node is MentionNode mentionNode && mentionNode.Type != MentionType.Meta)
{
if (mentionNode.Type == MentionType.User)
{
var user = _log.Mentionables.GetUser(mentionNode.Id);
buffer.Append($"@{user.Name}");
}
else if (mentionNode.Type == MentionType.Channel)
{
var channel = _log.Mentionables.GetChannel(mentionNode.Id);
buffer.Append($"#{channel.Name}");
}
else if (mentionNode.Type == MentionType.Role)
{
var role = _log.Mentionables.GetRole(mentionNode.Id);
buffer.Append($"@{role.Name}");
}
}
else if (node is EmojiNode emojiNode)
{
buffer.Append(emojiNode.IsCustomEmoji ? $":{emojiNode.Name}:" : node.Lexeme);
}
else
{
buffer.Append(node.Lexeme);
}
}
return buffer.ToString();
}
private string FormatMarkdownPlainText(string input)
=> FormatMarkdownPlainText(MarkdownParser.Parse(input));
private string FormatMarkdownHtml(IReadOnlyList<Node> nodes, int depth = 0)
{
var buffer = new StringBuilder();
foreach (var node in nodes)
{
if (node is TextNode textNode)
{
buffer.Append(textNode.Text.HtmlEncode());
}
else if (node is FormattedNode formattedNode)
{
var innerHtml = FormatMarkdownHtml(formattedNode.Children, depth + 1);
if (formattedNode.Formatting == TextFormatting.Bold)
buffer.Append($"<strong>{innerHtml}</strong>");
else if (formattedNode.Formatting == TextFormatting.Italic)
buffer.Append($"<em>{innerHtml}</em>");
else if (formattedNode.Formatting == TextFormatting.Underline)
buffer.Append($"<u>{innerHtml}</u>");
else if (formattedNode.Formatting == TextFormatting.Strikethrough)
buffer.Append($"<s>{innerHtml}</s>");
else if (formattedNode.Formatting == TextFormatting.Spoiler)
buffer.Append($"<span class=\"spoiler\">{innerHtml}</span>");
}
else if (node is InlineCodeBlockNode inlineCodeBlockNode)
{
buffer.Append($"<span class=\"pre pre--inline\">{inlineCodeBlockNode.Code.HtmlEncode()}</span>");
}
else if (node is MultilineCodeBlockNode multilineCodeBlockNode)
{
// Set language class for syntax highlighting
var languageCssClass = multilineCodeBlockNode.Language.IsNotBlank()
? "language-" + multilineCodeBlockNode.Language
: null;
buffer.Append(
$"<div class=\"pre pre--multiline {languageCssClass}\">{multilineCodeBlockNode.Code.HtmlEncode()}</div>");
}
else if (node is MentionNode mentionNode)
{
if (mentionNode.Type == MentionType.Meta)
{
buffer.Append($"<span class=\"mention\">@{mentionNode.Id.HtmlEncode()}</span>");
}
else if (mentionNode.Type == MentionType.User)
{
var user = _log.Mentionables.GetUser(mentionNode.Id);
buffer.Append($"<span class=\"mention\" title=\"{user.FullName}\">@{user.Name.HtmlEncode()}</span>");
}
else if (mentionNode.Type == MentionType.Channel)
{
var channel = _log.Mentionables.GetChannel(mentionNode.Id);
buffer.Append($"<span class=\"mention\">#{channel.Name.HtmlEncode()}</span>");
}
else if (mentionNode.Type == MentionType.Role)
{
var role = _log.Mentionables.GetRole(mentionNode.Id);
buffer.Append($"<span class=\"mention\">@{role.Name.HtmlEncode()}</span>");
}
}
else if (node is EmojiNode emojiNode)
{
// Get emoji image URL
var emojiImageUrl = new Emoji(emojiNode.Id, emojiNode.Name, emojiNode.IsAnimated).ImageUrl;
// Emoji can be jumboable if it's the only top-level node
var jumboableCssClass = depth == 0 && nodes.Count == 1
? "emoji--large"
: null;
buffer.Append($"<img class=\"emoji {jumboableCssClass}\" alt=\"{emojiNode.Name}\" title=\"{emojiNode.Name}\" src=\"{emojiImageUrl}\" />");
}
else if (node is LinkNode linkNode)
{
var escapedUrl = Uri.EscapeUriString(linkNode.Url);
buffer.Append($"<a href=\"{escapedUrl}\">{linkNode.Title.HtmlEncode()}</a>");
}
}
return buffer.ToString();
}
private string FormatMarkdownHtml(string input)
=> FormatMarkdownHtml(MarkdownParser.Parse(input));
private string FormatMarkdown(string input)
{
return _format == ExportFormat.HtmlDark || _format == ExportFormat.HtmlLight
? FormatMarkdownHtml(input)
: FormatMarkdownPlainText(input);
}
public ScriptObject GetScriptObject()
{
// Create instance
var scriptObject = new ScriptObject();
// Import model
scriptObject.SetValue("Model", _log, true);
// Import functions
scriptObject.Import(nameof(GroupMessages), new Func<IEnumerable<Message>, IEnumerable<MessageGroup>>(GroupMessages));
scriptObject.Import(nameof(Format), new Func<IFormattable, string, string>(Format));
scriptObject.Import(nameof(FormatDate), new Func<DateTime, string>(FormatDate));
scriptObject.Import(nameof(FormatMarkdown), new Func<string, string>(FormatMarkdown));
return scriptObject;
}
}
}
}
@@ -1,130 +0,0 @@
using System.Collections.Generic;
using System.IO;
using DiscordChatExporter.Core.Models;
using Scriban;
using Scriban.Runtime;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Services
{
public partial class ExportService
{
private readonly SettingsService _settingsService;
public ExportService(SettingsService settingsService)
{
_settingsService = settingsService;
}
private void ExportChatLogSingle(ChatLog chatLog, string filePath, ExportFormat format)
{
// Create template loader
var loader = new TemplateLoader();
// Get template
var templateCode = loader.Load(format);
var template = Template.Parse(templateCode);
// Create template context
var context = new TemplateContext
{
TemplateLoader = loader,
MemberRenamer = m => m.Name,
MemberFilter = m => true,
LoopLimit = int.MaxValue,
StrictVariables = true
};
// Create template model
var templateModel = new TemplateModel(format, chatLog,
_settingsService.DateFormat, _settingsService.MessageGroupLimit);
context.PushGlobal(templateModel.GetScriptObject());
// Create directory
var dirPath = Path.GetDirectoryName(filePath);
if (dirPath.IsNotBlank())
Directory.CreateDirectory(dirPath);
// Render output
using (var output = File.CreateText(filePath))
{
// Configure output
context.PushOutput(new TextWriterOutput(output));
// Render output
context.Evaluate(template.Page);
}
}
private void ExportChatLogPartitioned(IReadOnlyList<ChatLog> partitions, string filePath, ExportFormat format)
{
// Split file path into components
var dirPath = Path.GetDirectoryName(filePath);
var fileNameWithoutExt = Path.GetFileNameWithoutExtension(filePath);
var fileExt = Path.GetExtension(filePath);
// Export each partition separately
var partitionNumber = 1;
foreach (var partition in partitions)
{
// Compose new file name
var partitionFilePath = $"{fileNameWithoutExt} [{partitionNumber} of {partitions.Count}]{fileExt}";
// Compose full file path
if (dirPath.IsNotBlank())
partitionFilePath = Path.Combine(dirPath, partitionFilePath);
// Export
ExportChatLogSingle(partition, partitionFilePath, format);
// Increment partition number
partitionNumber++;
}
}
private IReadOnlyList<ChatLog> SplitIntoPartitions(ChatLog chatLog, int partitionLimit)
{
var result = new List<ChatLog>();
// Loop through all messages with an increment of partition limit
for (var i = 0; i < chatLog.Messages.Count; i += partitionLimit)
{
// Calculate how many messages left in total
var remainingMessageCount = chatLog.Messages.Count - i;
// Decide how many messages are going into this partition
// Each partition will have the same number of messages except the last one that might have fewer (all remaining messages)
var partitionMessageCount = partitionLimit.ClampMax(remainingMessageCount);
// Get messages that belong to this partition
var partitionMessages = new List<Message>();
for (var j = i; j < i + partitionMessageCount; j++)
partitionMessages.Add(chatLog.Messages[j]);
// Create a partition and add to list
var partition = new ChatLog(chatLog.Guild, chatLog.Channel, chatLog.From, chatLog.To, partitionMessages,
chatLog.Mentionables);
result.Add(partition);
}
return result;
}
public void ExportChatLog(ChatLog chatLog, string filePath, ExportFormat format,
int? partitionLimit = null)
{
// If partitioning is disabled or there are fewer messages in chat log than the limit - process it without partitioning
if (partitionLimit == null || partitionLimit <= 0 || chatLog.Messages.Count <= partitionLimit)
{
ExportChatLogSingle(chatLog, filePath, format);
}
// Otherwise split into partitions and export separately
else
{
var partitions = SplitIntoPartitions(chatLog, partitionLimit.Value);
ExportChatLogPartitioned(partitions, filePath, format);
}
}
}
}
@@ -1,60 +0,0 @@
using System;
using System.Threading.Tasks;
using Onova;
using Onova.Services;
namespace DiscordChatExporter.Core.Services
{
public class UpdateService
{
private readonly SettingsService _settingsService;
private readonly IUpdateManager _updateManager = new UpdateManager(
new GithubPackageResolver("Tyrrrz", "DiscordChatExporter", "DiscordChatExporter.zip"),
new ZipPackageExtractor());
private Version _updateVersion;
private bool _updaterLaunched;
public UpdateService(SettingsService settingsService)
{
_settingsService = settingsService;
}
public async Task<Version> CheckPrepareUpdateAsync()
{
// If auto-update is disabled - don't check for updates
if (!_settingsService.IsAutoUpdateEnabled)
return null;
// Cleanup leftover files
_updateManager.Cleanup();
// Check for updates
var check = await _updateManager.CheckForUpdatesAsync();
if (!check.CanUpdate)
return null;
// Prepare the update
if (!_updateManager.IsUpdatePrepared(check.LastVersion))
await _updateManager.PrepareUpdateAsync(check.LastVersion);
return _updateVersion = check.LastVersion;
}
public void FinalizeUpdate(bool needRestart)
{
// Check if an update is pending
if (_updateVersion == null)
return;
// Check if the updater has already been launched
if (_updaterLaunched)
return;
// Launch the updater
_updateManager.LaunchUpdater(_updateVersion, needRestart);
_updaterLaunched = true;
}
}
}
+1 -1
View File
@@ -14,7 +14,7 @@ namespace DiscordChatExporter.Gui
{
base.ConfigureIoC(builder);
// Autobind services in the .Core assembly
// Autobind the .Services assembly
builder.Autobind(typeof(DataService).Assembly);
// Bind settings as singleton
@@ -0,0 +1,28 @@
using System;
using System.Globalization;
using System.Windows.Data;
namespace DiscordChatExporter.Gui.Converters
{
[ValueConversion(typeof(DateTimeOffset), typeof(DateTime))]
public class DateTimeOffsetToDateTimeConverter : IValueConverter
{
public static DateTimeOffsetToDateTimeConverter Instance { get; } = new DateTimeOffsetToDateTimeConverter();
public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
{
if (value is DateTimeOffset date)
return date.DateTime;
return null;
}
public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
{
if (value is DateTime date)
return new DateTimeOffset(date);
return null;
}
}
}
@@ -12,8 +12,10 @@ namespace DiscordChatExporter.Gui.Converters
public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
{
var format = value as ExportFormat?;
return format?.GetDisplayName();
if (value is ExportFormat format)
return format.GetDisplayName();
return null;
}
public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
@@ -8,13 +8,14 @@
<OutputType>WinExe</OutputType>
<RootNamespace>DiscordChatExporter.Gui</RootNamespace>
<AssemblyName>DiscordChatExporter</AssemblyName>
<TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
<TargetFrameworkVersion>v4.6</TargetFrameworkVersion>
<FileAlignment>512</FileAlignment>
<ProjectTypeGuids>{60dc8134-eba5-43b8-bcc9-bb4bc16c2548};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
<WarningLevel>4</WarningLevel>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<NuGetPackageImportStamp>
</NuGetPackageImportStamp>
<TargetFrameworkProfile />
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<PlatformTarget>AnyCPU</PlatformTarget>
@@ -58,6 +59,7 @@
<Compile Include="Behaviors\ChannelViewModelMultiSelectionListBoxBehavior.cs" />
<Compile Include="Behaviors\MultiSelectionListBoxBehavior.cs" />
<Compile Include="Bootstrapper.cs" />
<Compile Include="Converters\DateTimeOffsetToDateTimeConverter.cs" />
<Compile Include="Converters\ExportFormatToStringConverter.cs" />
<Compile Include="ViewModels\Components\ChannelViewModel.cs" />
<Compile Include="ViewModels\Components\GuildViewModel.cs" />
@@ -96,9 +98,13 @@
<Resource Include="..\favicon.ico" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\DiscordChatExporter.Core\DiscordChatExporter.Core.csproj">
<ProjectReference Include="..\DiscordChatExporter.Core.Models\DiscordChatExporter.Core.Models.csproj">
<Project>{67a9d184-4656-4ce1-9d75-bddcbcafb200}</Project>
<Name>DiscordChatExporter.Core.Models</Name>
</ProjectReference>
<ProjectReference Include="..\DiscordChatExporter.Core.Services\DiscordChatExporter.Core.Services.csproj">
<Project>{707c0cd0-a7e0-4cab-8db9-07a45cb87377}</Project>
<Name>DiscordChatExporter.Core</Name>
<Name>DiscordChatExporter.Core.Services</Name>
</ProjectReference>
</ItemGroup>
<ItemGroup>
@@ -142,9 +148,8 @@
<Version>2.0.20525</Version>
</PackageReference>
<PackageReference Include="Tyrrrz.Extensions">
<Version>1.5.1</Version>
<Version>1.6.1</Version>
</PackageReference>
</ItemGroup>
<ItemGroup />
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
</Project>
@@ -3,5 +3,5 @@
[assembly: AssemblyTitle("DiscordChatExporter")]
[assembly: AssemblyCompany("Tyrrrz")]
[assembly: AssemblyCopyright("Copyright (c) Alexey Golub")]
[assembly: AssemblyVersion("2.10.1")]
[assembly: AssemblyFileVersion("2.10.1")]
[assembly: AssemblyVersion("2.12.1")]
[assembly: AssemblyFileVersion("2.12.1")]
@@ -1,9 +1,9 @@
using System;
using System.Collections.Generic;
using System.Linq;
using DiscordChatExporter.Core.Helpers;
using DiscordChatExporter.Core.Models;
using DiscordChatExporter.Core.Services;
using DiscordChatExporter.Core.Services.Helpers;
using DiscordChatExporter.Gui.ViewModels.Components;
using DiscordChatExporter.Gui.ViewModels.Framework;
using Tyrrrz.Extensions;
@@ -28,9 +28,9 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
public ExportFormat SelectedFormat { get; set; } = ExportFormat.HtmlDark;
public DateTime? From { get; set; }
public DateTimeOffset? After { get; set; }
public DateTime? To { get; set; }
public DateTimeOffset? Before { get; set; }
public int? PartitionLimit { get; set; }
@@ -55,11 +55,11 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
_settingsService.LastExportFormat = SelectedFormat;
_settingsService.LastPartitionLimit = PartitionLimit;
// Clamp 'from' and 'to' values
if (From > To)
From = To;
if (To < From)
To = From;
// Clamp 'after' and 'before' values
if (After > Before)
After = Before;
if (Before < After)
Before = After;
// If single channel - prompt file path
if (IsSingleChannel)
@@ -68,7 +68,7 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
var channel = Channels.Single();
// Generate default file name
var defaultFileName = ExportHelper.GetDefaultExportFileName(SelectedFormat, Guild, channel, From, To);
var defaultFileName = ExportHelper.GetDefaultExportFileName(SelectedFormat, Guild, channel, After, Before);
// Generate filter
var ext = SelectedFormat.GetFileExtension();
@@ -85,7 +85,7 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
}
// If canceled - return
if (OutputPath.IsBlank())
if (OutputPath.IsNullOrWhiteSpace())
return;
// Close dialog
@@ -1,6 +1,5 @@
using DiscordChatExporter.Core.Services;
using DiscordChatExporter.Gui.ViewModels.Framework;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Gui.ViewModels.Dialogs
{
@@ -20,12 +19,6 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
set => _settingsService.DateFormat = value;
}
public int MessageGroupLimit
{
get => _settingsService.MessageGroupLimit;
set => _settingsService.MessageGroupLimit = value.ClampMin(0);
}
public SettingsViewModel(SettingsService settingsService)
{
_settingsService = settingsService;
@@ -4,10 +4,10 @@ using System.IO;
using System.Linq;
using System.Net;
using System.Reflection;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Helpers;
using DiscordChatExporter.Core.Models;
using DiscordChatExporter.Core.Services;
using DiscordChatExporter.Core.Services.Exceptions;
using DiscordChatExporter.Core.Services.Helpers;
using DiscordChatExporter.Gui.ViewModels.Components;
using DiscordChatExporter.Gui.ViewModels.Framework;
using Gress;
@@ -62,9 +62,9 @@ namespace DiscordChatExporter.Gui.ViewModels
// Update busy state when progress manager changes
ProgressManager.Bind(o => o.IsActive, (sender, args) => IsBusy = ProgressManager.IsActive);
ProgressManager.Bind(o => o.IsActive,
(sender, args) => IsProgressIndeterminate = ProgressManager.IsActive && ProgressManager.Progress <= 0);
(sender, args) => IsProgressIndeterminate = ProgressManager.IsActive && ProgressManager.Progress.IsEither(0, 1));
ProgressManager.Bind(o => o.Progress,
(sender, args) => IsProgressIndeterminate = ProgressManager.IsActive && ProgressManager.Progress <= 0);
(sender, args) => IsProgressIndeterminate = ProgressManager.IsActive && ProgressManager.Progress.IsEither(0, 1));
}
protected override async void OnViewLoaded()
@@ -122,7 +122,7 @@ namespace DiscordChatExporter.Gui.ViewModels
await _dialogManager.ShowDialogAsync(dialog);
}
public bool CanPopulateGuildsAndChannels => !IsBusy && TokenValue.IsNotBlank();
public bool CanPopulateGuildsAndChannels => !IsBusy && !TokenValue.IsNullOrWhiteSpace();
public async void PopulateGuildsAndChannels()
{
@@ -235,7 +235,7 @@ namespace DiscordChatExporter.Gui.ViewModels
}
}
public bool CanExportChannels => !IsBusy && SelectedChannels.NotNullAndAny();
public bool CanExportChannels => !IsBusy && !SelectedChannels.IsNullOrEmpty();
public async void ExportChannels()
{
@@ -267,7 +267,7 @@ namespace DiscordChatExporter.Gui.ViewModels
{
// Generate default file name
var fileName = ExportHelper.GetDefaultExportFileName(dialog.SelectedFormat, dialog.Guild,
channel, dialog.From, dialog.To);
channel, dialog.After, dialog.Before);
// Combine paths
filePath = Path.Combine(filePath, fileName);
@@ -275,10 +275,10 @@ namespace DiscordChatExporter.Gui.ViewModels
// Get chat log
var chatLog = await _dataService.GetChatLogAsync(token, dialog.Guild, channel,
dialog.From, dialog.To, operation);
dialog.After, dialog.Before, operation);
// Export
_exportService.ExportChatLog(chatLog, filePath, dialog.SelectedFormat,
await _exportService.ExportChatLogAsync(chatLog, filePath, dialog.SelectedFormat,
dialog.PartitionLimit);
// Notify completion
@@ -86,8 +86,8 @@
Margin="16,8"
materialDesign:HintAssist.Hint="From (optional)"
materialDesign:HintAssist.IsFloating="True"
DisplayDateEnd="{Binding To}"
SelectedDate="{Binding From}"
DisplayDateEnd="{Binding Before, Converter={x:Static converters:DateTimeOffsetToDateTimeConverter.Instance}}"
SelectedDate="{Binding After, Converter={x:Static converters:DateTimeOffsetToDateTimeConverter.Instance}}"
ToolTip="If this is set, only messages sent after this date will be exported" />
<DatePicker
Grid.Row="0"
@@ -95,8 +95,8 @@
Margin="16,8"
materialDesign:HintAssist.Hint="To (optional)"
materialDesign:HintAssist.IsFloating="True"
DisplayDateStart="{Binding From}"
SelectedDate="{Binding To}"
DisplayDateStart="{Binding After, Converter={x:Static converters:DateTimeOffsetToDateTimeConverter.Instance}}"
SelectedDate="{Binding Before, Converter={x:Static converters:DateTimeOffsetToDateTimeConverter.Instance}}"
ToolTip="If this is set, only messages sent before this date will be exported" />
</Grid>
@@ -25,13 +25,6 @@
materialDesign:HintAssist.IsFloating="True"
Text="{Binding DateFormat}" />
<!-- Message group limit -->
<TextBox
Margin="16,8"
materialDesign:HintAssist.Hint="Message group limit"
materialDesign:HintAssist.IsFloating="True"
Text="{Binding MessageGroupLimit}" />
<!-- Auto-updates -->
<DockPanel LastChildFill="False">
<TextBlock
+30 -18
View File
@@ -1,7 +1,7 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 15
VisualStudioVersion = 15.0.27130.2026
# Visual Studio Version 16
VisualStudioVersion = 16.0.28729.10
MinimumVisualStudioVersion = 10.0.40219.1
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{EA305DD5-1F98-415D-B6C4-65053A58F914}"
ProjectSection(SolutionItems) = preProject
@@ -10,36 +10,48 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
Readme.md = Readme.md
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DiscordChatExporter.Core.Markdown", "DiscordChatExporter.Core.Markdown\DiscordChatExporter.Core.Markdown.csproj", "{14D02A08-E820-4012-B805-663B9A3D73E9}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DiscordChatExporter.Core.Models", "DiscordChatExporter.Core.Models\DiscordChatExporter.Core.Models.csproj", "{67A9D184-4656-4CE1-9D75-BDDCBCAFB200}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DiscordChatExporter.Core.Rendering", "DiscordChatExporter.Core.Rendering\DiscordChatExporter.Core.Rendering.csproj", "{D33F7443-4EEB-4E53-99BE-6045A62FC8C8}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DiscordChatExporter.Core.Services", "DiscordChatExporter.Core.Services\DiscordChatExporter.Core.Services.csproj", "{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DiscordChatExporter.Gui", "DiscordChatExporter.Gui\DiscordChatExporter.Gui.csproj", "{732A67AF-93DE-49DF-B10F-FD74710B7863}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DiscordChatExporter.Core", "DiscordChatExporter.Core\DiscordChatExporter.Core.csproj", "{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DiscordChatExporter.Cli", "DiscordChatExporter.Cli\DiscordChatExporter.Cli.csproj", "{D08624B6-3081-4BCB-91F8-E9832FACC6CE}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DiscordChatExporter.Core.Markdown", "DiscordChatExporter.Core.Markdown\DiscordChatExporter.Core.Markdown.csproj", "{14D02A08-E820-4012-B805-663B9A3D73E9}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Debug|Any CPU.Build.0 = Debug|Any CPU
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Release|Any CPU.ActiveCfg = Release|Any CPU
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Release|Any CPU.Build.0 = Release|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Debug|Any CPU.Build.0 = Debug|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Release|Any CPU.ActiveCfg = Release|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Release|Any CPU.Build.0 = Release|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Release|Any CPU.Build.0 = Release|Any CPU
{14D02A08-E820-4012-B805-663B9A3D73E9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{14D02A08-E820-4012-B805-663B9A3D73E9}.Debug|Any CPU.Build.0 = Debug|Any CPU
{14D02A08-E820-4012-B805-663B9A3D73E9}.Release|Any CPU.ActiveCfg = Release|Any CPU
{14D02A08-E820-4012-B805-663B9A3D73E9}.Release|Any CPU.Build.0 = Release|Any CPU
{67A9D184-4656-4CE1-9D75-BDDCBCAFB200}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{67A9D184-4656-4CE1-9D75-BDDCBCAFB200}.Debug|Any CPU.Build.0 = Debug|Any CPU
{67A9D184-4656-4CE1-9D75-BDDCBCAFB200}.Release|Any CPU.ActiveCfg = Release|Any CPU
{67A9D184-4656-4CE1-9D75-BDDCBCAFB200}.Release|Any CPU.Build.0 = Release|Any CPU
{D33F7443-4EEB-4E53-99BE-6045A62FC8C8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D33F7443-4EEB-4E53-99BE-6045A62FC8C8}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D33F7443-4EEB-4E53-99BE-6045A62FC8C8}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D33F7443-4EEB-4E53-99BE-6045A62FC8C8}.Release|Any CPU.Build.0 = Release|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Debug|Any CPU.Build.0 = Debug|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Release|Any CPU.ActiveCfg = Release|Any CPU
{707C0CD0-A7E0-4CAB-8DB9-07A45CB87377}.Release|Any CPU.Build.0 = Release|Any CPU
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Debug|Any CPU.Build.0 = Debug|Any CPU
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Release|Any CPU.ActiveCfg = Release|Any CPU
{732A67AF-93DE-49DF-B10F-FD74710B7863}.Release|Any CPU.Build.0 = Release|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D08624B6-3081-4BCB-91F8-E9832FACC6CE}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
+21 -12
View File
@@ -1,13 +1,22 @@
FROM mono:5
WORKDIR /root/build
COPY DiscordChatExporter.sln favicon.ico ./
COPY DiscordChatExporter.Core DiscordChatExporter.Core
COPY DiscordChatExporter.Core.Markdown DiscordChatExporter.Core.Markdown
COPY DiscordChatExporter.Cli DiscordChatExporter.Cli
RUN msbuild ./DiscordChatExporter.Cli/DiscordChatExporter.Cli.csproj /t:Restore
RUN msbuild ./DiscordChatExporter.Cli/DiscordChatExporter.Cli.csproj /p:Configuration=Release
# Build
FROM microsoft/dotnet:2.1-sdk AS build
WORKDIR /src
FROM mono:5
COPY --from=0 /root/build/DiscordChatExporter.Cli/bin/Release/net461 /root/bin
WORKDIR /a
ENTRYPOINT ["mono", "/root/bin/DiscordChatExporter.Cli.exe"]
COPY favicon.ico ./
COPY DiscordChatExporter.Core.Markdown DiscordChatExporter.Core.Markdown
COPY DiscordChatExporter.Core.Models DiscordChatExporter.Core.Models
COPY DiscordChatExporter.Core.Rendering DiscordChatExporter.Core.Rendering
COPY DiscordChatExporter.Core.Services DiscordChatExporter.Core.Services
COPY DiscordChatExporter.Cli DiscordChatExporter.Cli
RUN dotnet publish DiscordChatExporter.Cli -c Release -f netcoreapp2.1
# Run
FROM microsoft/dotnet:2.1-runtime AS run
WORKDIR /app
COPY --from=build /src/DiscordChatExporter.Cli/bin/Release/netcoreapp2.1/publish ./
WORKDIR /app/out
ENTRYPOINT ["dotnet", "/app/DiscordChatExporter.Cli.dll"]
+8 -2
View File
@@ -3,14 +3,17 @@
[![Build](https://img.shields.io/appveyor/ci/Tyrrrz/DiscordChatExporter/master.svg)](https://ci.appveyor.com/project/Tyrrrz/DiscordChatExporter)
[![Release](https://img.shields.io/github/release/Tyrrrz/DiscordChatExporter.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
[![Downloads](https://img.shields.io/github/downloads/Tyrrrz/DiscordChatExporter/total.svg)](https://github.com/Tyrrrz/DiscordChatExporter/releases)
[![Donate](https://img.shields.io/badge/patreon-donate-yellow.svg)](https://patreon.com/tyrrrz)
[![Donate](https://img.shields.io/badge/buymeacoffee-donate-yellow.svg)](https://buymeacoffee.com/tyrrrz)
DiscordChatExporter can be used to export message history from a [Discord](https://discordapp.com) channel to a file. It works for both direct message chats and guild chats, supports markdown, message grouping, embeds, attachments, mentions, reactions and other features. It works with both user and bot tokens, supports multiple output formats and allows you to trim messages by dates.
## Download
- [Stable releases](https://github.com/Tyrrrz/DiscordChatExporter/releases)
- [Chocolatey](https://chocolatey.org/packages/DiscordChatExporter): `choco install discordchatexporter`
- [Docker](https://hub.docker.com/r/tyrrrz/discordchatexporter): `docker pull tyrrrz/discordchatexporter`
- [Continuous integration](https://ci.appveyor.com/project/Tyrrrz/DiscordChatExporter)
- [Docker](https://hub.docker.com/r/tyrrrz/discordchatexporter): `docker pull tyrrrz/discordchatexporter` (only CLI version)
## Features
@@ -34,9 +37,12 @@ DiscordChatExporter can be used to export message history from a [Discord](https
- [Scriban](https://github.com/lunet-io/scriban)
- [CommandLineParser](https://github.com/commandlineparser/commandline)
- [Ookii.Dialogs](https://github.com/caioproiete/ookii-dialogs-wpf)
- [ByteSize](https://github.com/omar/ByteSize)
- [Failsafe](https://github.com/Tyrrrz/Failsafe)
- [Gress](https://github.com/Tyrrrz/Gress)
- [Onova](https://github.com/Tyrrrz/Onova)
- [Tyrrrz.Extensions](https://github.com/Tyrrrz/Extensions)
- [Tyrrrz.Settings](https://github.com/Tyrrrz/Settings)
## Donate
If you really like my projects and want to support me, consider donating to me on [Patreon](https://patreon.com/tyrrrz) or [BuyMeACoffee](https://buymeacoffee.com/tyrrrz). All donations are optional and are greatly appreciated. 🙏
+3 -1
View File
@@ -9,4 +9,6 @@ function Replace-TextInFile {
}
Replace-TextInFile "$PSScriptRoot\DiscordChatExporter.Cli\DiscordChatExporter.Cli.csproj" '(?<=<Version>)(.*?)(?=</Version>)' $newVersion
Replace-TextInFile "$PSScriptRoot\DiscordChatExporter.Gui\Properties\AssemblyInfo.cs" '(?<=Assembly.*?Version\(")(.*?)(?="\)\])' $newVersion
Replace-TextInFile "$PSScriptRoot\DiscordChatExporter.Gui\Properties\AssemblyInfo.cs" '(?<=Assembly.*?Version\(")(.*?)(?="\)\])' $newVersion
Replace-TextInFile "$PSScriptRoot\Deploy\Choco\discordchatexporter.nuspec" '(?<=<version>)(.*?)(?=</version>)' $newVersion
Replace-TextInFile "$PSScriptRoot\Deploy\Choco\tools\chocolateyinstall.ps1" '(?<=download/)(.*?)(?=/)' $newVersion

Some files were not shown because too many files have changed in this diff Show More