Compare commits

...

52 Commits

Author SHA1 Message Date
Tyrrrz de773467b6 Update version 2021-07-19 01:58:23 +03:00
Tyrrrz 92766b580b [HTML] Show "Click to see attachment" on replies to attachment or embed messages without content
Closes #634
2021-07-19 01:12:30 +03:00
Tyrrrz abf7498667 Add support for timestamp markers
Closes #637
2021-07-18 16:04:39 +03:00
Tyrrrz ae42554621 Render voice channel mentions
Closes #636
2021-07-18 02:19:50 +03:00
Tyrrrz b58e67776c [HTML] Improve how the reference symbol looks
Closes #650
2021-07-18 01:09:52 +03:00
Tyrrrz 7fc6e0cfe0 Update readme 2021-07-18 00:19:25 +03:00
Tyrrrz 35264e8a12 Improve "Requested resource does not exist" exception message to give slightly more context 2021-07-18 00:15:06 +03:00
Tyrrrz 2ab6773c17 [GUI] Show error in a dialog when pull or export fails, instead of crashing the whole app 2021-07-18 00:02:28 +03:00
Tyrrrz 650c55bbd1 Refactor 2021-07-17 23:53:13 +03:00
Cliff J e1726683f8 Hide author icon when loading returns an error (#644)
Fixes #643
2021-07-13 14:56:36 -07:00
Yudi 70a1755c50 Update the in-app instructions for getting user token (#639)
Co-authored-by: virtuallyvlad <37453320+virtuallyvlad@users.noreply.github.com>
Co-authored-by: Alexey Golub <tyrrrrrr@gmail.com>
2021-07-13 07:26:26 -07:00
Tyrrrz 524b30b09b Update CliFx to fix NotSupportedException
Fixes #631
2021-07-09 22:53:12 +03:00
Lucas LaBuff d39fc212b6 Filter which messages are exported (#622) 2021-07-07 12:14:40 -07:00
Tyrrrz 95cd6cb50c Further cleanup issue forms 2021-06-24 20:46:42 +03:00
Tyrrrz 79a9ac0462 Refactor issue forms and add pull request template 2021-06-24 19:29:15 +03:00
Tyrrrz 5b5a53dbf4 [HTML] More template cleanup 2021-06-22 20:03:30 +03:00
Tyrrrz 2b0fa4213b [HTML] Cleanup templates slightly 2021-06-22 14:16:08 +03:00
Tyrrrz fd9d367bad [HTML] Cleanup formatting in stylesheet 2021-06-21 20:52:30 +03:00
Tyrrrz dbc3209160 [HTML] Merge all static stylesheets into a single dynamic stylesheet 2021-06-21 20:44:36 +03:00
Tyrrrz 3bcde0c534 [HTML] Move stylesheets into their own template 2021-06-21 19:57:15 +03:00
Tyrrrz 903882e35b Fix typos in changelog 2021-06-20 23:06:39 +03:00
Luke 3c6d9d74a4 Lazy load images (#613) 2021-06-20 05:50:14 -07:00
Tyrrrz 1e53f50ca3 [HTML] Simplify disabling jumbo in referenced message content 2021-06-19 23:09:07 +03:00
Tyrrrz cba5a934df Update version 2021-06-19 20:50:22 +03:00
Tyrrrz cd05058cf7 Prefer proxy URLs for embed content everywhere 2021-06-19 20:25:50 +03:00
Tyrrrz e9c3872701 [HTML] Prevent click events on elements inside referenced message
Closes #545
2021-06-19 20:17:36 +03:00
Tyrrrz 88c4330a66 [HTML] Link to pre-resized avatar image for better image quality 2021-06-19 20:15:18 +03:00
Tyrrrz 998bdaf2d4 [HTML] Use svg variants of Twemoji for better resize quality 2021-06-19 19:57:29 +03:00
Tyrrrz 24a80f915f [HTML] Recognize standard emoji by code and show emoji code in tooltips
Closes #549
Closes #599
2021-06-19 19:49:53 +03:00
Tyrrrz de57cd714d [HTML] Prepend "Message sent: " to message timestamp tooltips 2021-06-19 18:56:47 +03:00
Tyrrrz f9ab269b29 [HTML] Don't jumbo emojis in referenced messages
Closes #544
2021-06-19 15:49:16 +03:00
Tyrrrz 33feaccee7 Only use DotnetRuntimeBootstrapper in release build 2021-06-17 22:08:11 +03:00
Tyrrrz 4fd6f8272e [HTML] Update styling to match Discord 2021-06-16 18:15:58 +03:00
Tyrrrz b28f029382 Refactor YouTube embeds 2021-06-16 17:54:35 +03:00
Tyrrrz 29552be6e5 Cleanup 2021-06-16 16:52:18 +03:00
Tyrrrz 9491e18e2f Update NuGet packages 2021-06-15 22:41:06 +03:00
quentinmay 4b1c9660be Render YouTube iframe for applicable video embeds (#607) 2021-06-15 07:19:19 -07:00
Tyrrrz 950625e671 [HTML] Show timestamps of individual messages
Closes #536
2021-06-14 22:55:42 +03:00
Tyrrrz ce2dedff6b [HTML] Use a clickable <div> instead of <a> for reply reference link
Fixes #546
2021-06-14 21:33:20 +03:00
Tyrrrz baac419538 [HTML] Fix embed footer styling 2021-06-14 20:48:20 +03:00
Tyrrrz bd40dbde48 Don't throw if file dates cannot be modified when downloading media
Fixes #585
2021-06-14 20:43:44 +03:00
Tyrrrz cbe4bc593c [HTML] Prefer proxy URLs for resolving embed media
Closes #590
2021-06-14 20:38:19 +03:00
Tyrrrz cb32cee5f5 Update DotnetRuntimeBootstrapper 2021-06-14 04:51:32 +03:00
Tyrrrz 82f364a887 Update readme 2021-06-08 18:04:19 +03:00
Tyrrrz 152e8c8dcb Update version 2021-06-08 17:54:58 +03:00
John Zabroski 260139ab25 Fix .NET v3.1 to say .NET Core v3.1 in readme (#605) 2021-06-08 07:25:12 -07:00
Tyrrrz 11e9c0f15c Downgrade MDIX 2021-06-08 01:32:21 +03:00
Tyrrrz ac809d1a3f Update packages 2021-06-08 01:23:00 +03:00
Tyrrrz 36fa346299 Use DotnetRuntimeBootstrapper 2021-06-08 00:53:41 +03:00
Tyrrrz 8c3a5bd096 Update issue templates 2021-05-12 21:38:12 +03:00
Tyrrrz 021682899a [CLI] Move Markup.Escape(...) to ConsoleExtensions 2021-04-26 19:47:25 +03:00
Lucas LaBuff 0ec6027a93 Escape brackets in category name in CLI (#558) 2021-04-26 09:37:37 -07:00
105 changed files with 10847 additions and 1442 deletions
+30 -8
View File
@@ -1,25 +1,31 @@
name: "\U0001F41E Bug report"
name: 🐞 Bug report
description: Report broken functionality.
labels: [bug]
body:
- type: markdown
attributes:
value: |
- Please check existing issues (both opened and closed) to ensure that this bug hasn't been reported before.
- Refer to the [contribution guidelines](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/Contributing.md) for info on how to file a good issue.
- If you want to ask a question instead of reporting a bug, use [**discussions**](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
🧐 **Guidelines:**
- Search through [existing issues](https://github.com/Tyrrrz/DiscordChatExporter/issues?q=is%3Aissue) first to ensure that this bug has not been reported before.
- Write a descriptive title for your issue. Avoid generic or vague titles such as "Something's not working" or "A couple of problems".
- Keep your issue focused on one single problem. If you have multiple bug reports, please create separate issues for each of them.
- Provide as much context as possible in the details section. Include screenshots, screen recordings, links, references, or anything else you may consider relevant.
- If you want to ask a question instead of reporting a bug, please use [discussions](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
- type: input
attributes:
label: Version
description: "Which version(s) of DiscordChatExporter does this bug affect?"
description: Which version of DiscordChatExporter does this bug affect?
placeholder: ver X.Y.Z
validations:
required: true
- type: dropdown
attributes:
label: Flavor
description: "Which flavor(s) of DiscordChatExporter does this bug affect?"
description: Does this bug affect the GUI version, the CLI version, or both?
multiple: true
options:
- GUI (Graphical User Interface)
@@ -27,16 +33,32 @@ body:
validations:
required: true
- type: dropdown
attributes:
label: Export format
description: Does this bug affect specific export format(s)? If not applicable, leave this field blank.
multiple: true
options:
- HTML
- TXT
- JSON
- CSV
- type: textarea
attributes:
label: Details
description: "Clear and thorough explanation of the bug. If relevant, include screenshots or screen recordings."
description: Clear and thorough explanation of the bug.
placeholder: I was doing X expecting Y to happen, but Z happened instead.
validations:
required: true
- type: textarea
attributes:
label: Steps to reproduce
description: "Minimum steps required to reproduce the bug."
description: Minimum steps required to reproduce the bug.
placeholder: |
- Step 1
- Step 2
- Step 3
validations:
required: true
+4 -4
View File
@@ -1,8 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: "\U0001F4D6 Read the wiki"
- name: 📖 Wiki
url: https://github.com/Tyrrrz/DiscordChatExporter/wiki
about: Find answers to most common questions.
- name: "\U0001F5E8 Ask a question"
about: Find usage guides and frequently asked questions.
- name: 🗨 Discussions
url: https://github.com/Tyrrrz/DiscordChatExporter/discussions/new
about: Please ask and answer questions here.
about: Ask and answer questions.
+22 -6
View File
@@ -1,26 +1,42 @@
name: "\U00002728 Feature request"
name: Feature request
description: Request a new feature.
labels: [enhancement]
body:
- type: markdown
attributes:
value: |
- Please check existing issues (both opened and closed) to ensure that this feature hasn't been requested before.
- Refer to the [contribution guidelines](https://github.com/Tyrrrz/DiscordChatExporter/blob/master/Contributing.md) for info on how to file a good issue.
- If you want to ask a question instead of requesting a feature, use [**discussions**](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
🧐 **Guidelines:**
- Search through [existing issues](https://github.com/Tyrrrz/DiscordChatExporter/issues?q=is%3Aissue) first to ensure that this feature has not been requested before.
- Write a descriptive title for your issue. Avoid generic or vague titles such as "Some suggestions" or "Ideas for improvement".
- Keep your issue focused on one single problem. If you have multiple feature requests, please create separate issues for each of them.
- Provide as much context as possible in the details section. Include screenshots, screen recordings, links, references, or anything else you may consider relevant.
- If you want to ask a question instead of requesting a feature, please use [discussions](https://github.com/Tyrrrz/DiscordChatExporter/discussions/new) instead.
- type: dropdown
attributes:
label: Flavor
description: "Which flavor(s) of DiscordChatExporter does this feature request apply to?"
description: Is this feature request related to specific flavor(s) of DiscordChatExporter? If not applicable, leave this field blank.
multiple: true
options:
- GUI (Graphical User Interface)
- CLI (Command Line Interface)
- type: dropdown
attributes:
label: Export format
description: Is this feature request related to specific export format(s)? If not applicable, leave this field blank.
multiple: true
options:
- HTML
- TXT
- JSON
- CSV
- type: textarea
attributes:
label: Details
description: "Clear and thorough explanation of the feature you have in mind."
description: Clear and thorough explanation of the feature you have in mind.
validations:
required: true
+17
View File
@@ -0,0 +1,17 @@
<!--
**Important:**
Please make sure that there is an existing issue that describes the problem solved by your pull request. If there isn't one, consider creating it first.
An open issue offers a good place to iron out requirements, discuss possible solutions, and ask questions.
Remember to also:
- Keep your pull request focused and as small as possible. If you want to contribute multiple unrelated changes, please create separate pull requests for them.
- Follow the coding style and conventions already established by the project. When in doubt about which style to use, ask in the comments to your pull request.
- If you want to start a discussion regarding a specific change you've made, add a review comment to your own code. This can be used to highlight something important or to seek further input from others.
-->
<!-- Please specify the issue addressed by this pull request -->
Closes #ISSUE_NUMBER
+47 -10
View File
@@ -1,3 +1,40 @@
### v2.29 (19-Jul-2021)
- Added support for message filtering, which is accessible as an additional export parameter in the GUI and as the `--filter` option in the CLI. Message filters use a special syntax similar to Discord's search query syntax to allow specifying which messages should be included in the export. For example, you can limit the export to only include messages that contain image attachments (`has:image`) or messages from a certain user (`from:Tyrrrz`) or messages containing certain words (`hello world`), etc. You can also combine filters together (`hi mentions:Tyrrrz`, `from:Tyrrrz | from:96-LB`, `(mentions:Tyrrrz | from:Tyrrrz) has:image`) or negate them (`-has:link`). More thorough documentation coming soon. (Thanks [@Lucas LaBuff](https://github.com/96-LB))
- Added support for rendering mentions of voice channels, in addition to text channels.
- Added support for rendering unix timestamp markers. Currently, style markers are ignored and timestamps are always rendered with the format configured in settings (or provided via `--date-format` option).
- Updated in-app instructions for obtaining a user token. (Thanks [@Yudi](https://github.com/Yudi) and [@virtuallyvlad](https://github.com/virtuallyvlad))
- [HTML] Improved visual appearance of the reference symbol (Г-shaped line in replies) to make it more closely resemble Discord's styling.
- [HTML] Added more specific fallback text used when rendering referenced messages that contain no text content but have attachments or embeds.
- [HTML] Fixed an issue where a broken embed author icon occupied visual space, ruining the embed layout. (Thanks [@Cliff J.](https://github.com/CliffJameston))
- [GUI] Improved handling for fatal errors that may occur when attempting to export a channel. These errors are now displayed in a popup, allowing the user to try again instead of having the application crash completely.
- [CLI] Fixed an issue where an attempt to export a channel resulted in `NotSupportedException` when the default encoding was set to `UTF8`.
- [CLI] Fixed an issue where providing an incorrectly formatted partition limit via `-p|--partition` option resulted in a non-descriptive and unhelpful error.
- Improved "Requested resource does not exist" error message by including the ID of the requested resource.
### v2.28 (19-Jun-2021)
- [HTML] Added special casing for YouTube video embeds to render them directly using an embedded player. (Thanks [@quentinmay](https://github.com/quentinmay))
- [HTML] Added support for rendering standard emoji by code. Such emoji may sometimes be found in messages sent by bots or through webhooks. (Thanks [@CanePlayz](https://github.com/CanePlayz) and [@Lucas LaBuff](https://github.com/96-LB))
- [HTML] Changed tooltips on standard emoji to show emoji code instead of their raw string representations. (Thanks [@CanePlayz](https://github.com/CanePlayz) and [@Lucas LaBuff](https://github.com/96-LB))
- [HTML] Added tooltips on individual messages to show when those messages were sent.
- [HTML] Updated colors and minor styling elements to match Discord's new style direction after rebranding.
- [HTML] Updated Twemoji URLs to use SVG image variants for better rendering quality.
- [HTML] Changed user avatar URLs to include predetermined size query parameter for better rendering quality.
- Changed embed rendering logic to prefer Discord-proxied URLs for external content, in order to avoid unnecessary HTTP requests to third parties.
- Fixed an issue where exporting with media sometimes resulted in a crash due to lack of permissions to change file attributes. These errors are now ignored, which means that the creation, write, and access dates of downloaded files may sometimes not match the dates provided by Discord's CDN.
- [HTML] Fixed styling issues when rendering embed footers.
- [HTML] Fixed an issue where links didn't render properly inside referenced messages.
- [HTML] Fixed an issue where standalone emoji were incorrectly enlarged inside referenced messages.
- [HTML] Fixed an issue where clicking on a referenced message revealed spoilers inside of it.
- [GUI] Fixed an issue where the list of missing components, reported by the prerequisite check at application startup, included Windows updates that were not applicable to the user's system due to the fact that other superseding updates have already been installed. This issue only affected users running DiscordChatExporter on Windows 7.
- [GUI] Fixed a few other minor issues related to the prerequisite installation window.
### v2.27.1 (08-Jun-2021)
- [GUI] Application will now detect if the required .NET Runtime or any of its prerequisites are missing and prompt the user to download and install them automatically. **Experimental feature, please test it out and report any issues you may find!**
- [CLI] Fixed an issue where the application crashed when exporting a channel that had square brackets as part of its name or category name. (Thanks [@Lucas LaBuff](https://github.com/96-LB))
### v2.27 (24-Apr-2021)
- Added partitioning by file size. You can now use values such as `10mb` to indicate a size-based cut off point, in addition to values like `10` to indicate a number of messages. (Thanks [@Andrew Kolos](https://github.com/andrewkolos))
@@ -73,7 +110,7 @@
### v2.21 (18-Jul-2020)
- Added a new option that enables self-contained exports for all output formats. You can turn it on in the export setup dialog in GUI or using the `--media` option in CLI. When using this, the application will additionally download any media content directly referenced from the exported file instead of linking back to Discord CDN. The files which are downloaded include: guild icons, user avatars, attachments, embedded images, reaction emojis. Note that only files which are actually referenced by the export are downloaded, which means that, for example, user avatars will not be downloaded when using plain text export format. This option is not meant to enable complete offline viewing for HTML exports, but rather to make it easier to archive media content that may eventually get deleted from Discord servers. Also keep in mind that this option may make the export drastically slower and the total file size larger.
- Added a new option that enables self-contained exports for all output formats. You can turn it on in the export setup dialog in GUI or using the `--media` option in CLI. When using this, the application will additionally download any media content directly referenced from the exported file instead of linking back to Discord CDN. The files which are downloaded include: guild icons, user avatars, attachments, embedded images, reaction emoji. Note that only files which are actually referenced by the export are downloaded, which means that, for example, user avatars will not be downloaded when using plain text export format. This option is not meant to enable complete offline viewing for HTML exports, but rather to make it easier to archive media content that may eventually get deleted from Discord servers. Also keep in mind that this option may make the export drastically slower and the total file size larger.
- Changed "discordapp.com" to "discord.com" where applicable as Discord is migrating to a new domain. CDN will remain on "cdn.discordapp.com" for the foreseeable future.
Note that all existing and current HTML exports will likely not render accurately because Discord enabled CORS for their font resources, which prevents them from loading locally. Please refer to [issue #322](https://github.com/Tyrrrz/DiscordChatExporter/issues/322) for discussion on this topic.
@@ -112,7 +149,7 @@ Note that all existing and current HTML exports will likely not render accuratel
- Fixed an issue where an empty file was produced when exporting a channel with no messages (for specified period). With the new behavior, no file will be created and instead a message will be shown to the user informing of the failure.
- [HTML/TXT] Added message count to the bottom of the file. This number shows how many messages were exported as part of the current file. Prior to 2.16 this number was present along with the rest of the metadata at the top, then removed due to changes in the exporter, and now brought back at the end of the file instead.
- [HTML] Fixed an issue where some emojis were not properly rendered.
- [HTML] Fixed an issue where some emoji were not properly rendered.
- [GUI] Added a setting that lets you configure whether to remember the last used token. Previously it was the default behavior, but now you can disable it if you don't want the token persisted on your system.
### v2.16 (08-Dec-2019)
@@ -125,7 +162,7 @@ Note that all existing and current HTML exports will likely not render accuratel
- Fixed various issues that resulted in exceptions during export process.
- [HTML/TXT] Removed message count from the metadata. Due to the fact that the messages are exported as they are streamed, total number of messages is not known ahead of time.
- [HTML] Changed default color of the embed color pill to match the theme.
- [HTML] Changed emoji parser to be less greedy. As a result it should match fewer character sequences that look like emojis but really aren't, but on the other hand it might miss some actual emojis. This means that some standard emojis (i.e. not custom server emojis) may not look like in Discord. This is a compromise I'm willing to take because detecting emojis in text is really hard to get right and not worth it at all.
- [HTML] Changed emoji parser to be less greedy. As a result it should match fewer character sequences that look like emoji but really aren't, but on the other hand it might miss some actual emoji. This means that some standard emoji (i.e. not custom server emoji) may not look like in Discord. This is a compromise I'm willing to take because detecting emoji in text is really hard to get right and not worth it at all.
- [HTML] Some other minor styling adjustments.
- [CSV] Replaced semicolons (;) with (,) as separators.
@@ -166,8 +203,8 @@ Please note that CSV export is most likely not going to receive future updates a
### v2.12.1 (12-Apr-2019)
- [GUI] Fixed an issue where the app crashed when trying to export multiple channels.
- [HTML] Fixed an issue where some of the emojis were rendered via Twemoji while Discord renders them as plain text.
- [HTML] Fixed an issue where emojis weren't jumbo if there were more than one of them.
- [HTML] Fixed an issue where some of the emoji were rendered via Twemoji while Discord renders them as plain text.
- [HTML] Fixed an issue where emoji weren't jumbo if there were more than one of them.
### v2.12 (11-Apr-2019)
@@ -189,15 +226,15 @@ Please note that CSV export is most likely not going to receive future updates a
### v2.10.1 (06-Mar-2019)
- [HTML] Fixed an issue where multiple emojis on a single line would get rendered as one emoji.
- [HTML] Fixed an issue where multiple emoji on a single line would get rendered as one emoji.
- [HTML] Fixed an issue where an open square bracket character would incorrectly render as an emoji.
- [HTML] Added alt text to emoji images so that they will be rendered using default font when Twemoji images are not available.
### v2.10 (03-Mar-2019)
- [HTML] Reworked the entire markdown parsing engine which fixes numerous rendering inconsistencies (over 15 issues fixed).
- [HTML] Added support for animated emojis.
- [HTML] Standard emojis are now rendered using Twemoji, the same image set that Discord uses.
- [HTML] Added support for animated emoji.
- [HTML] Standard emoji are now rendered using Twemoji, the same image set that Discord uses.
- [HTML] Fixed an issue where video attachments would render incorrectly.
- [HTML] Minor fixes in styling.
- [GUI] Aggregated export progress is now shown in the taskbar.
@@ -257,7 +294,7 @@ Please note that CSV export is most likely not going to receive future updates a
- Added support for reactions (HTML only).
- Selected date range is now shown in the export (HTML and PlainText).
- Added placeholders for guild icons to display while the icons are loading.
- Added support for jumbo emojis (custom emojis only).
- Added support for jumbo emoji (custom emoji only).
- Added basic retry policy for 429 error responses.
- Added validation for from/to dates in the UI which prevents choosing invalid date ranges.
- Added an error message when trying to export a deleted channel.
@@ -281,7 +318,7 @@ Please note that CSV export is most likely not going to receive future updates a
- Improved message date filtering, it's now marginally faster.
- Fixed underscores not recognized as italics in markdown.
- Added support for custom emojis.
- Added support for custom emoji.
- Added support for user and role mentions.
- Added support for channel mentions.
- Fixed text in pre blocks not being wrapped correctly.
-56
View File
@@ -1,56 +0,0 @@
# Contributing
DiscordChatExporter accepts contributions in the form of issues and pull requests.
## Creating issues
If you have a feature suggestion or want to report a bug, you are welcome to create an issue.
Guidelines:
- Avoid creating an issue if a similar one already exists. Look through existing open and closed issues first.
- Keep your issue focused on one specific problem. If you have multiple suggestions or bug reports, please create separate issues for them.
- Provide a descriptive title for your issue. Don't use generic titles like "A couple suggestions" or "Not working".
- Provide more context in the body of the issue. If relevant, attach screenshots or screen recordings.
- Remain civil and respectful when participating in discussions.
## Creating pull requests
If you want to contribute code to the project, you can do so by creating a pull request.
Guidelines:
- Make sure that there is an existing issue that describes the problem solved by your pull request. If there isn't one, please create it first.
- Add a comment to the issue indicating that you want to submit a pull request for it. This can be a good opportunity to discuss possible solutions, finalize requirements, or ask questions.
- Keep your pull request focused and as small as possible. If you want to contribute multiple unrelated changes, please create separate pull requests for them.
- Follow the coding style and conventions already established by the project. When in doubt which style to use, ask in comments to your pull request.
- If you want to start a discussion regarding a specific change you made, add a review comment to your own code. This can be used to highlight something important or to seek input from others.
## Building the project locally
Prerequisites:
- [.NET 5.0 SDK](https://dotnet.microsoft.com/download/dotnet/5.0)
- [.NET 3.1 SDK](https://dotnet.microsoft.com/download/dotnet/3.1) (temporarily as the app still targets .NET 3.1, but requires .NET 5.0 to build)
- _(Optional)_ C#/.NET IDE, such as [JetBrains Rider](https://www.jetbrains.com/rider), [VS Code](https://code.visualstudio.com/docs/languages/csharp), or [Visual Studio](https://visualstudio.microsoft.com).
To build the entire solution run the following command in the root of the repository:
```sh
> dotnet build
```
This will generate runtime artifacts for each project:
```plaintext
./DiscordChatExporter.Gui/bin/[Debug|Release]/[runtime]/*
./DiscordChatExporter.Cli/bin/[Debug|Release]/[runtime]/*
```
You can also build and run a specific project directly.
To do that, navigate to its directory and use `dotnet run`:
```sh
> cd DiscordChatExporter.Gui
> dotnet run
```
+2 -2
View File
@@ -2,7 +2,7 @@
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<Version>2.27</Version>
<Version>2.29</Version>
<Company>Tyrrrz</Company>
<Copyright>Copyright (c) Alexey Golub</Copyright>
<LangVersion>preview</LangVersion>
@@ -10,4 +10,4 @@
<WarningsAsErrors>nullable</WarningsAsErrors>
</PropertyGroup>
</Project>
</Project>
@@ -12,6 +12,7 @@ using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exceptions;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Filtering;
using DiscordChatExporter.Core.Exporting.Partitioning;
using DiscordChatExporter.Core.Utils.Extensions;
using Tyrrrz.Extensions;
@@ -32,8 +33,11 @@ namespace DiscordChatExporter.Cli.Commands.Base
[CommandOption("before", Description = "Only include messages sent before this date or message ID.")]
public Snowflake? Before { get; init; }
[CommandOption("partition", 'p', Description = "Split output into partitions, each limited to this number of messages (e.g. 100) or file size (e.g. 10mb).")]
public PartitionLimit PartitionLimit { get; init; } = NullPartitionLimit.Instance;
[CommandOption("partition", 'p', Description = "Split output into partitions, each limited to this number of messages (e.g. '100') or file size (e.g. '10mb').")]
public PartitionLimit PartitionLimit { get; init; } = PartitionLimit.Null;
[CommandOption("filter", Description = "Only include messages that satisfy this filter (e.g. 'from:foo#1234' or 'has:image').")]
public MessageFilter MessageFilter { get; init; } = MessageFilter.Null;
[CommandOption("parallel", Description = "Limits how many channels can be exported in parallel.")]
public int ParallelLimit { get; init; } = 1;
@@ -76,6 +80,7 @@ namespace DiscordChatExporter.Cli.Commands.Base
After,
Before,
PartitionLimit,
MessageFilter,
ShouldDownloadMedia,
ShouldReuseMedia,
DateFormat
@@ -128,8 +133,6 @@ namespace DiscordChatExporter.Cli.Commands.Base
{
throw new CommandException("Export failed.");
}
await console.Output.WriteLineAsync("Done.");
}
public override ValueTask ExecuteAsync(IConsole console)
@@ -31,6 +31,10 @@ namespace DiscordChatExporter.Cli.Commands
await foreach (var channel in Discord.GetGuildChannelsAsync(guild.Id))
{
// Skip non-text channels
if (!channel.IsTextChannel)
continue;
channels.Add(channel);
}
}
@@ -1,4 +1,5 @@
using System.Threading.Tasks;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
@@ -17,9 +18,10 @@ namespace DiscordChatExporter.Cli.Commands
// Get channel metadata
await console.Output.WriteLineAsync("Fetching channels...");
var channels = await Discord.GetGuildChannelsAsync(Guild.DirectMessages.Id);
var textChannels = channels.Where(c => c.IsTextChannel).ToArray();
// Export
await ExportAsync(console, channels);
await ExportAsync(console, textChannels);
}
}
}
@@ -1,4 +1,5 @@
using System.Threading.Tasks;
using System.Linq;
using System.Threading.Tasks;
using CliFx.Attributes;
using CliFx.Infrastructure;
using DiscordChatExporter.Cli.Commands.Base;
@@ -20,9 +21,10 @@ namespace DiscordChatExporter.Cli.Commands
// Get channel metadata
await console.Output.WriteLineAsync("Fetching channels...");
var channels = await Discord.GetGuildChannelsAsync(GuildId);
var textChannels = channels.Where(c => c.IsTextChannel).ToArray();
// Export
await ExportAsync(console, channels);
await ExportAsync(console, textChannels);
}
}
}
@@ -19,7 +19,13 @@ namespace DiscordChatExporter.Cli.Commands
{
var channels = await Discord.GetGuildChannelsAsync(GuildId);
foreach (var channel in channels.OrderBy(c => c.Category.Position).ThenBy(c => c.Name))
var textChannels = channels
.Where(c => c.IsTextChannel)
.OrderBy(c => c.Category.Position)
.ThenBy(c => c.Name)
.ToArray();
foreach (var channel in textChannels)
{
// Channel ID
await console.Output.WriteAsync(channel.Id.ToString());
@@ -16,7 +16,13 @@ namespace DiscordChatExporter.Cli.Commands
{
var channels = await Discord.GetGuildChannelsAsync(Guild.DirectMessages.Id);
foreach (var channel in channels.OrderBy(c => c.Name))
var textChannels = channels
.Where(c => c.IsTextChannel)
.OrderBy(c => c.Category.Position)
.ThenBy(c => c.Name)
.ToArray();
foreach (var channel in textChannels)
{
// Channel ID
await console.Output.WriteAsync(channel.Id.ToString());
@@ -1,4 +1,4 @@
using System;
using System;
using System.Threading.Tasks;
using CliFx;
using CliFx.Attributes;
@@ -16,10 +16,12 @@ namespace DiscordChatExporter.Cli.Commands
console.Output.WriteLine(" 1. Open Discord");
console.Output.WriteLine(" 2. Press Ctrl+Shift+I to show developer tools");
console.Output.WriteLine(" 3. Navigate to the Application tab");
console.Output.WriteLine(" 4. Select \"Local Storage\" > \"https://discord.com\" on the left");
console.Output.WriteLine(" 5. Press Ctrl+R to reload");
console.Output.WriteLine(" 6. Find \"token\" at the bottom and copy the value");
console.Output.WriteLine(" 3. Press Ctrl+Shift+M to toggle device toolbar");
console.Output.WriteLine(" 4. Navigate to the Application tab");
console.Output.WriteLine(" 5. On the left, expand Local Storage and select https://discord.com");
console.Output.WriteLine(" 6. Type \"token\" into the Filter box");
console.Output.WriteLine(" 7. If the token key does not appear, press Ctrl+R to reload");
console.Output.WriteLine(" 8. Copy the value of the token key");
console.Output.WriteLine(" * Automating user accounts is technically against TOS, use at your own risk.");
console.Output.WriteLine();
@@ -6,8 +6,8 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliFx" Version="2.0.4" />
<PackageReference Include="Spectre.Console" Version="0.39.0" />
<PackageReference Include="CliFx" Version="2.0.6" />
<PackageReference Include="Spectre.Console" Version="0.41.0" />
<PackageReference Include="Gress" Version="1.2.0" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.5" />
</ItemGroup>
@@ -33,7 +33,11 @@ namespace DiscordChatExporter.Cli.Utils.Extensions
string description,
Func<ProgressTask, ValueTask> performOperationAsync)
{
var progressTask = progressContext.AddTask(description, new ProgressTaskSettings {MaxValue = 1});
var progressTask = progressContext.AddTask(
// Don't recognize random square brackets as style tags
Markup.Escape(description),
new ProgressTaskSettings {MaxValue = 1}
);
try
{
@@ -2,8 +2,6 @@
namespace DiscordChatExporter.Core.Discord
{
public enum AuthTokenType { User, Bot }
public class AuthToken
{
public AuthTokenType Type { get; }
@@ -0,0 +1,8 @@
namespace DiscordChatExporter.Core.Discord
{
public enum AuthTokenType
{
User,
Bot
}
}
@@ -17,18 +17,19 @@ namespace DiscordChatExporter.Core.Discord.Data
public string FileName { get; }
public string FileExtension => Path.GetExtension(FileName);
public int? Width { get; }
public int? Height { get; }
public bool IsImage => ImageFileExtensions.Contains(Path.GetExtension(FileName));
public bool IsImage => ImageFileExtensions.Contains(FileExtension);
public bool IsVideo => VideoFileExtensions.Contains(Path.GetExtension(FileName));
public bool IsVideo => VideoFileExtensions.Contains(FileExtension);
public bool IsAudio => AudioFileExtensions.Contains(Path.GetExtension(FileName));
public bool IsAudio => AudioFileExtensions.Contains(FileExtension);
public bool IsSpoiler =>
(IsImage || IsVideo || IsAudio) && FileName.StartsWith("SPOILER_", StringComparison.Ordinal);
public bool IsSpoiler => FileName.StartsWith("SPOILER_", StringComparison.Ordinal);
public FileSize FileSize { get; }
@@ -7,32 +7,21 @@ using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Discord.Data
{
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
// Order of enum fields needs to match the order in the docs.
public enum ChannelType
{
GuildTextChat,
DirectTextChat,
GuildVoiceChat,
DirectGroupTextChat,
GuildCategory,
GuildNews,
GuildStore
}
// https://discord.com/developers/docs/resources/channel#channel-object
public partial class Channel : IHasId, IHasPosition
public partial class Channel : IHasId
{
public Snowflake Id { get; }
public ChannelType Type { get; }
public bool IsTextChannel =>
Type == ChannelType.GuildTextChat ||
Type == ChannelType.DirectTextChat ||
Type == ChannelType.DirectGroupTextChat ||
Type == ChannelType.GuildNews ||
Type == ChannelType.GuildStore;
public bool IsTextChannel => Type is
ChannelType.GuildTextChat or
ChannelType.DirectTextChat or
ChannelType.DirectGroupTextChat or
ChannelType.GuildNews or
ChannelType.GuildStore;
public bool IsVoiceChannel => !IsTextChannel;
public Snowflake GuildId { get; }
@@ -48,7 +37,7 @@ namespace DiscordChatExporter.Core.Discord.Data
Snowflake id,
ChannelType type,
Snowflake guildId,
ChannelCategory? category,
ChannelCategory category,
string name,
int? position,
string? topic)
@@ -56,14 +45,13 @@ namespace DiscordChatExporter.Core.Discord.Data
Id = id;
Type = type;
GuildId = guildId;
Category = category ?? GetFallbackCategory(type);
Category = category;
Name = name;
Position = position;
Topic = topic;
}
public override string ToString() => Name;
}
public partial class Channel
@@ -79,7 +67,7 @@ namespace DiscordChatExporter.Core.Discord.Data
ChannelType.GuildStore => "Store",
_ => "Default"
},
0
null
);
public static Channel Parse(JsonElement json, ChannelCategory? category = null, int? position = null)
@@ -87,23 +75,23 @@ namespace DiscordChatExporter.Core.Discord.Data
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);
var guildId = json.GetPropertyOrNull("guild_id")?.GetString().Pipe(Snowflake.Parse);
var topic = json.GetPropertyOrNull("topic")?.GetString();
var type = (ChannelType) json.GetProperty("type").GetInt32();
var name =
// Guild channel
json.GetPropertyOrNull("name")?.GetString() ??
// DM channel
json.GetPropertyOrNull("recipients")?.EnumerateArray().Select(User.Parse).Select(u => u.Name).JoinToString(", ") ??
// Fallback
id.ToString();
position ??= json.GetPropertyOrNull("position")?.GetInt32();
return new Channel(
id,
type,
guildId ?? Guild.DirectMessages.Id,
category ?? GetFallbackCategory(type),
name,
position,
position ?? json.GetPropertyOrNull("position")?.GetInt32(),
topic
);
}
@@ -1,13 +1,11 @@
using System.Linq;
using System.Text.Json;
using System.Text.Json;
using DiscordChatExporter.Core.Discord.Data.Common;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Discord.Data
{
public partial class ChannelCategory : IHasId, IHasPosition
public partial class ChannelCategory : IHasId
{
public Snowflake Id { get; }
@@ -23,7 +21,6 @@ namespace DiscordChatExporter.Core.Discord.Data
}
public override string ToString() => Name;
}
public partial class ChannelCategory
@@ -31,19 +28,18 @@ namespace DiscordChatExporter.Core.Discord.Data
public static ChannelCategory Parse(JsonElement json, int? position = null)
{
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);
position ??= json.GetPropertyOrNull("position")?.GetInt32();
var name = json.GetPropertyOrNull("name")?.GetString() ??
json.GetPropertyOrNull("recipients")?.EnumerateArray().Select(User.Parse).Select(u => u.Name).JoinToString(", ") ??
var name =
json.GetPropertyOrNull("name")?.GetString() ??
id.ToString();
return new ChannelCategory(
id,
name,
position
position ?? json.GetPropertyOrNull("position")?.GetInt32()
);
}
public static ChannelCategory Empty { get; } = new(Snowflake.Zero, "<unknown category>", 0);
}
}
}
@@ -0,0 +1,15 @@
namespace DiscordChatExporter.Core.Discord.Data
{
// https://discord.com/developers/docs/resources/channel#channel-object-channel-types
// Order of enum fields needs to match the order in the docs.
public enum ChannelType
{
GuildTextChat = 0,
DirectTextChat,
GuildVoiceChat,
DirectGroupTextChat,
GuildCategory,
GuildNews,
GuildStore
}
}
@@ -1,7 +0,0 @@
namespace DiscordChatExporter.Core.Discord.Data.Common
{
public interface IHasPosition
{
int? Position { get; }
}
}
@@ -55,6 +55,8 @@ namespace DiscordChatExporter.Core.Discord.Data
Footer = footer;
}
public YouTubeVideoEmbedProjection? TryGetYouTubeVideo() => YouTubeVideoEmbedProjection.TryResolve(this);
public override string ToString() => Title ?? "<untitled embed>";
}
@@ -63,10 +65,10 @@ namespace DiscordChatExporter.Core.Discord.Data
public static Embed Parse(JsonElement json)
{
var title = json.GetPropertyOrNull("title")?.GetString();
var description = json.GetPropertyOrNull("description")?.GetString();
var url = json.GetPropertyOrNull("url")?.GetString();
var timestamp = json.GetPropertyOrNull("timestamp")?.GetDateTimeOffset();
var color = json.GetPropertyOrNull("color")?.GetInt32().Pipe(System.Drawing.Color.FromArgb).ResetAlpha();
var description = json.GetPropertyOrNull("description")?.GetString();
var author = json.GetPropertyOrNull("author")?.Pipe(EmbedAuthor.Parse);
var thumbnail = json.GetPropertyOrNull("thumbnail")?.Pipe(EmbedImage.Parse);
@@ -12,11 +12,14 @@ namespace DiscordChatExporter.Core.Discord.Data
public string? IconUrl { get; }
public EmbedAuthor(string? name, string? url, string? iconUrl)
public string? IconProxyUrl { get; }
public EmbedAuthor(string? name, string? url, string? iconUrl, string? iconProxyUrl)
{
Name = name;
Url = url;
IconUrl = iconUrl;
IconProxyUrl = iconProxyUrl;
}
public override string ToString() => Name ?? "<unnamed author>";
@@ -29,8 +32,9 @@ namespace DiscordChatExporter.Core.Discord.Data
var name = json.GetPropertyOrNull("name")?.GetString();
var url = json.GetPropertyOrNull("url")?.GetString();
var iconUrl = json.GetPropertyOrNull("icon_url")?.GetString();
var iconProxyUrl = json.GetPropertyOrNull("proxy_icon_url")?.GetString();
return new EmbedAuthor(name, url, iconUrl);
return new EmbedAuthor(name, url, iconUrl, iconProxyUrl);
}
}
}
@@ -10,10 +10,13 @@ namespace DiscordChatExporter.Core.Discord.Data
public string? IconUrl { get; }
public EmbedFooter(string text, string? iconUrl)
public string? IconProxyUrl { get; }
public EmbedFooter(string text, string? iconUrl, string? iconProxyUrl)
{
Text = text;
IconUrl = iconUrl;
IconProxyUrl = iconProxyUrl;
}
public override string ToString() => Text;
@@ -25,8 +28,9 @@ namespace DiscordChatExporter.Core.Discord.Data
{
var text = json.GetProperty("text").GetString();
var iconUrl = json.GetPropertyOrNull("icon_url")?.GetString();
var iconProxyUrl = json.GetPropertyOrNull("proxy_icon_url")?.GetString();
return new EmbedFooter(text, iconUrl);
return new EmbedFooter(text, iconUrl, iconProxyUrl);
}
}
}
@@ -8,13 +8,16 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public string? Url { get; }
public string? ProxyUrl { get; }
public int? Width { get; }
public int? Height { get; }
public EmbedImage(string? url, int? width, int? height)
public EmbedImage(string? url, string? proxyUrl, int? width, int? height)
{
Url = url;
ProxyUrl = proxyUrl;
Height = height;
Width = width;
}
@@ -25,10 +28,11 @@ namespace DiscordChatExporter.Core.Discord.Data
public static EmbedImage Parse(JsonElement json)
{
var url = json.GetPropertyOrNull("url")?.GetString();
var proxyUrl = json.GetPropertyOrNull("proxy_url")?.GetString();
var width = json.GetPropertyOrNull("width")?.GetInt32();
var height = json.GetPropertyOrNull("height")?.GetInt32();
return new EmbedImage(url, width, height);
return new EmbedImage(url, proxyUrl, width, height);
}
}
}
+26 -31
View File
@@ -1,7 +1,7 @@
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Linq;
using System.Text.Json;
using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Reading;
using Tyrrrz.Extensions;
@@ -10,21 +10,27 @@ namespace DiscordChatExporter.Core.Discord.Data
// https://discord.com/developers/docs/resources/emoji#emoji-object
public partial class Emoji
{
// Only present on custom emoji
public string? Id { get; }
// Name of custom emoji (e.g. LUL) or actual representation of standard emoji (e.g. 🙂)
public string Name { get; }
// Name of custom emoji (e.g. LUL) or name of standard emoji (e.g. slight_smile)
public string Code => !string.IsNullOrWhiteSpace(Id)
? Name
: EmojiIndex.TryGetCode(Name) ?? Name;
public bool IsAnimated { get; }
public string ImageUrl { get; }
public Emoji(string? id, string name, bool isAnimated)
public Emoji(string? id, string name, bool isAnimated, string imageUrl)
{
Id = id;
Name = name;
IsAnimated = isAnimated;
ImageUrl = GetImageUrl(id, name, isAnimated);
ImageUrl = imageUrl;
}
public override string ToString() => Name;
@@ -32,39 +38,26 @@ namespace DiscordChatExporter.Core.Discord.Data
public partial class Emoji
{
private static IEnumerable<Rune> GetRunes(string emoji)
{
var lastIndex = 0;
while (lastIndex < emoji.Length && Rune.TryGetRuneAt(emoji, lastIndex, out var rune))
{
// Skip variant selector rune
if (rune.Value != 0xfe0f)
yield return rune;
lastIndex += rune.Utf16SequenceLength;
}
}
private static string GetTwemojiName(IEnumerable<Rune> runes) =>
runes.Select(r => r.Value.ToString("x")).JoinToString("-");
private static string GetTwemojiName(string name) => name
.GetRunes()
// Variant selector rune is skipped in Twemoji names
.Where(r => r.Value != 0xfe0f)
.Select(r => r.Value.ToString("x"))
.JoinToString("-");
public static string GetImageUrl(string? id, string name, bool isAnimated)
{
// Custom emoji
if (!string.IsNullOrWhiteSpace(id))
{
// Animated
if (isAnimated)
return $"https://cdn.discordapp.com/emojis/{id}.gif";
// Non-animated
return $"https://cdn.discordapp.com/emojis/{id}.png";
return isAnimated
? $"https://cdn.discordapp.com/emojis/{id}.gif"
: $"https://cdn.discordapp.com/emojis/{id}.png";
}
// Standard emoji
var emojiRunes = GetRunes(name).ToArray();
var twemojiName = GetTwemojiName(emojiRunes);
return $"https://twemoji.maxcdn.com/2/72x72/{twemojiName}.png";
var twemojiName = GetTwemojiName(name);
return $"https://twemoji.maxcdn.com/2/svg/{twemojiName}.svg";
}
public static Emoji Parse(JsonElement json)
@@ -73,7 +66,9 @@ namespace DiscordChatExporter.Core.Discord.Data
var name = json.GetProperty("name").GetString();
var isAnimated = json.GetPropertyOrNull("animated")?.GetBoolean() ?? false;
return new Emoji(id, name, isAnimated);
var imageUrl = GetImageUrl(id, name, isAnimated);
return new Emoji(id, name, isAnimated, imageUrl);
}
}
}
@@ -19,10 +19,10 @@ namespace DiscordChatExporter.Core.Discord.Data
public IReadOnlyList<Snowflake> RoleIds { get; }
public Member(User user, string? nick, IReadOnlyList<Snowflake> roleIds)
public Member(User user, string nick, IReadOnlyList<Snowflake> roleIds)
{
User = user;
Nick = nick ?? user.Name;
Nick = nick;
RoleIds = roleIds;
}
@@ -33,7 +33,7 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public static Member CreateForUser(User user) => new(
user,
null,
user.Name,
Array.Empty<Snowflake>()
);
@@ -43,12 +43,12 @@ namespace DiscordChatExporter.Core.Discord.Data
var nick = json.GetPropertyOrNull("nick")?.GetString();
var roleIds =
json.GetPropertyOrNull("roles")?.EnumerateArray().Select(j => j.GetString().Pipe(Snowflake.Parse)).ToArray() ??
json.GetPropertyOrNull("roles")?.EnumerateArray().Select(j => j.GetString()).Select(Snowflake.Parse).ToArray() ??
Array.Empty<Snowflake>();
return new Member(
user,
nick,
nick ?? user.Name,
roleIds
);
}
@@ -100,7 +100,7 @@ namespace DiscordChatExporter.Core.Discord.Data
var type = (MessageType) json.GetProperty("type").GetInt32();
var isPinned = json.GetPropertyOrNull("pinned")?.GetBoolean() ?? false;
var messageReference = json.GetPropertyOrNull("message_reference")?.Pipe(MessageReference.Parse);
var referencedMessage = json.GetPropertyOrNull("referenced_message")?.Pipe(Message.Parse);
var referencedMessage = json.GetPropertyOrNull("referenced_message")?.Pipe(Parse);
var content = type switch
{
@@ -23,8 +23,8 @@ namespace DiscordChatExporter.Core.Discord.Data
{
public static Reaction Parse(JsonElement json)
{
var count = json.GetProperty("count").GetInt32();
var emoji = json.GetProperty("emoji").Pipe(Emoji.Parse);
var count = json.GetProperty("count").GetInt32();
return new Reaction(emoji, count);
}
@@ -40,7 +40,8 @@ namespace DiscordChatExporter.Core.Discord.Data
var name = json.GetProperty("name").GetString();
var position = json.GetProperty("position").GetInt32();
var color = json.GetPropertyOrNull("color")?
var color = json
.GetPropertyOrNull("color")?
.GetInt32()
.Pipe(System.Drawing.Color.FromArgb)
.ResetAlpha()
@@ -49,19 +49,19 @@ namespace DiscordChatExporter.Core.Discord.Data
{
// Animated
if (avatarHash.StartsWith("a_", StringComparison.Ordinal))
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.gif";
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.gif?size=40";
// Non-animated
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.png";
return $"https://cdn.discordapp.com/avatars/{id}/{avatarHash}.png?size=40";
}
public static User Parse(JsonElement json)
{
var id = json.GetProperty("id").GetString().Pipe(Snowflake.Parse);
var isBot = json.GetPropertyOrNull("bot")?.GetBoolean() ?? false;
var discriminator = json.GetProperty("discriminator").GetString().Pipe(int.Parse);
var name = json.GetProperty("username").GetString();
var avatarHash = json.GetProperty("avatar").GetString();
var isBot = json.GetPropertyOrNull("bot")?.GetBoolean() ?? false;
var avatarUrl = !string.IsNullOrWhiteSpace(avatarHash)
? GetAvatarUrl(id, avatarHash)
@@ -0,0 +1,61 @@
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Discord.Data
{
public partial class YouTubeVideoEmbedProjection
{
public string VideoId { get; }
public string Url { get; }
public YouTubeVideoEmbedProjection(string videoId, string url)
{
VideoId = videoId;
Url = url;
}
public override string ToString() => Url;
}
public partial class YouTubeVideoEmbedProjection
{
// Adapted from YoutubeExplode
// https://github.com/Tyrrrz/YoutubeExplode/blob/5be164be20019783913f76fcc98f18c65aebe9f0/YoutubeExplode/Videos/VideoId.cs#L34-L64
private static string? TryParseVideoId(string embedUrl)
{
// Regular URL
// https://www.youtube.com/watch?v=yIVRs6YSbOM
var regularMatch = Regex.Match(embedUrl, @"youtube\..+?/watch.*?v=(.*?)(?:&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(regularMatch))
return regularMatch;
// Short URL
// https://youtu.be/yIVRs6YSbOM
var shortMatch = Regex.Match(embedUrl, @"youtu\.be/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(shortMatch))
return shortMatch;
// Embed URL
// https://www.youtube.com/embed/yIVRs6YSbOM
var embedMatch = Regex.Match(embedUrl, @"youtube\..+?/embed/(.*?)(?:\?|&|/|$)").Groups[1].Value;
if (!string.IsNullOrWhiteSpace(embedMatch))
return embedMatch;
return null;
}
public static YouTubeVideoEmbedProjection? TryResolve(Embed embed)
{
if (string.IsNullOrWhiteSpace(embed.Url))
return null;
var videoId = TryParseVideoId(embed.Url);
if (string.IsNullOrWhiteSpace(videoId))
return null;
var url = $"https://www.youtube.com/embed/{videoId}";
return new YouTubeVideoEmbedProjection(videoId, url);
}
}
}
@@ -11,7 +11,6 @@ using DiscordChatExporter.Core.Utils;
using DiscordChatExporter.Core.Utils.Extensions;
using JsonExtensions.Http;
using JsonExtensions.Reading;
using Tyrrrz.Extensions;
namespace DiscordChatExporter.Core.Discord
{
@@ -29,7 +28,9 @@ namespace DiscordChatExporter.Core.Discord
}
public DiscordClient(AuthToken token)
: this(Http.Client, token) {}
: this(Http.Client, token)
{
}
private async ValueTask<HttpResponseMessage> GetResponseAsync(string url) =>
await Http.ResponsePolicy.ExecuteAsync(async () =>
@@ -50,7 +51,7 @@ namespace DiscordChatExporter.Core.Discord
{
HttpStatusCode.Unauthorized => DiscordChatExporterException.Unauthorized(),
HttpStatusCode.Forbidden => DiscordChatExporterException.Forbidden(),
HttpStatusCode.NotFound => DiscordChatExporterException.NotFound(),
HttpStatusCode.NotFound => DiscordChatExporterException.NotFound(url),
_ => DiscordChatExporterException.FailedHttpRequest(response)
};
}
@@ -64,7 +65,7 @@ namespace DiscordChatExporter.Core.Discord
return response.IsSuccessStatusCode
? await response.Content.ReadAsJsonAsync()
: (JsonElement?) null;
: null;
}
public async IAsyncEnumerable<Guild> GetUserGuildsAsync()
@@ -118,32 +119,29 @@ namespace DiscordChatExporter.Core.Discord
{
var response = await GetJsonResponseAsync($"guilds/{guildId}/channels");
var orderedResponse = response
var responseOrdered = response
.EnumerateArray()
.OrderBy(j => j.GetProperty("position").GetInt32())
.ThenBy(j => ulong.Parse(j.GetProperty("id").GetString()))
.ThenBy(j => Snowflake.Parse(j.GetProperty("id").GetString()))
.ToArray();
var categories = orderedResponse
.Where(j => j.GetProperty("type").GetInt32() == (int)ChannelType.GuildCategory)
var categories = responseOrdered
.Where(j => j.GetProperty("type").GetInt32() == (int) ChannelType.GuildCategory)
.Select((j, index) => ChannelCategory.Parse(j, index + 1))
.ToDictionary(j => j.Id.ToString());
.ToDictionary(j => j.Id.ToString(), StringComparer.Ordinal);
var position = 0;
foreach (var channelJson in orderedResponse)
foreach (var channelJson in responseOrdered)
{
var parentId = channelJson.GetPropertyOrNull("parent_id")?.GetString();
var category = !string.IsNullOrWhiteSpace(parentId)
? categories.GetValueOrDefault(parentId)
: null;
var channel = Channel.Parse(channelJson, category, position);
// Skip non-text channels
if (!channel.IsTextChannel)
continue;
position++;
yield return channel;
@@ -178,15 +176,12 @@ namespace DiscordChatExporter.Core.Discord
var response = await GetJsonResponseAsync($"channels/{channelId}");
return ChannelCategory.Parse(response);
}
/***
* In some cases, the Discord API returns an empty body when requesting some channel category info.
* Instead, we use an empty channel category as a fallback.
*/
// In some cases, the Discord API returns an empty body when requesting channel category.
// Instead, we use an empty channel category as a fallback.
catch (DiscordChatExporterException)
{
return ChannelCategory.Empty;
}
}
public async ValueTask<Channel> GetChannelAsync(Snowflake channelId)
@@ -221,7 +216,7 @@ namespace DiscordChatExporter.Core.Discord
IProgress<double>? progress = null)
{
// Get the last message in the specified range.
// This snapshots the boundaries, which means that messages posted after the exported started
// This snapshots the boundaries, which means that messages posted after the export started
// will not appear in the output.
// Additionally, it provides the date of the last message, which is used to calculate progress.
var lastMessage = await TryGetLastMessageAsync(channelId, before);
@@ -271,7 +266,7 @@ namespace DiscordChatExporter.Core.Discord
progress.Report(exportedDuration / totalDuration);
}
// Avoid division by zero if all messages have the exact same timestamp
// (which can happen easily if there's only one message in the channel)
// (which may be the case if there's only one message in the channel)
else
{
progress.Report(1);
@@ -284,4 +279,4 @@ namespace DiscordChatExporter.Core.Discord
}
}
}
}
}
@@ -10,8 +10,9 @@ namespace DiscordChatExporter.Core.Discord
public Snowflake(ulong value) => Value = value;
public DateTimeOffset ToDate() =>
DateTimeOffset.FromUnixTimeMilliseconds((long) ((Value >> 22) + 1420070400000UL)).ToLocalTime();
public DateTimeOffset ToDate() => DateTimeOffset.FromUnixTimeMilliseconds(
(long) ((Value >> 22) + 1420070400000UL)
).ToLocalTime();
public override string ToString() => Value.ToString(CultureInfo.InvariantCulture);
}
@@ -53,9 +54,11 @@ namespace DiscordChatExporter.Core.Discord
public static Snowflake Parse(string str) => Parse(str, null);
}
public partial struct Snowflake : IEquatable<Snowflake>
public partial struct Snowflake : IComparable<Snowflake>, IEquatable<Snowflake>
{
public bool Equals(Snowflake other) => Value == other.Value;
public int CompareTo(Snowflake other) => Value.CompareTo(other.Value);
public bool Equals(Snowflake other) => CompareTo(other) == 0;
public override bool Equals(object? obj) => obj is Snowflake other && Equals(other);
@@ -5,19 +5,15 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="JsonExtensions" Version="1.0.1" />
<PackageReference Include="JsonExtensions" Version="1.1.0" />
<PackageReference Include="MiniRazor.CodeGen" Version="2.1.2" />
<PackageReference Include="Polly" Version="7.2.2" />
<PackageReference Include="Superpower" Version="2.3.0" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.5" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Exporting\Writers\Html\Core.css" />
<EmbeddedResource Include="Exporting\Writers\Html\Dark.css" />
<EmbeddedResource Include="Exporting\Writers\Html\Light.css" />
<AdditionalFiles Include="Exporting\Writers\Html\PreambleTemplate.cshtml" IsRazorTemplate="true" />
<AdditionalFiles Include="Exporting\Writers\Html\PostambleTemplate.cshtml" IsRazorTemplate="true" />
<AdditionalFiles Include="Exporting\Writers\Html\MessageGroupTemplate.cshtml" IsRazorTemplate="true" />
<AdditionalFiles Include="Exporting\Writers\Html\*.cshtml" IsRazorTemplate="true" />
</ItemGroup>
</Project>
@@ -36,8 +36,8 @@ Failed to perform an HTTP request.
internal static DiscordChatExporterException Forbidden() =>
new("Access is forbidden.");
internal static DiscordChatExporterException NotFound() =>
new("Requested resource does not exist.");
internal static DiscordChatExporterException NotFound(string resourceId) =>
new($"Requested resource ({resourceId}) does not exist.");
internal static DiscordChatExporterException ChannelIsEmpty() =>
new("No messages found for the specified period.");
@@ -39,6 +39,10 @@ namespace DiscordChatExporter.Core.Exporting
var encounteredUsers = new HashSet<User>(IdBasedEqualityComparer.Instance);
await foreach (var message in _discord.GetMessagesAsync(request.Channel.Id, request.After, request.Before, progress))
{
// Skips any messages that fail to pass the supplied filter
if (!request.MessageFilter.Filter(message))
continue;
// Resolve members for referenced users
foreach (var referencedUser in message.MentionedUsers.Prepend(message.Author))
{
@@ -79,7 +79,7 @@ namespace DiscordChatExporter.Core.Exporting
: filePath;
// HACK: for HTML, we need to format the URL properly
if (Request.Format == ExportFormat.HtmlDark || Request.Format == ExportFormat.HtmlLight)
if (Request.Format is ExportFormat.HtmlDark or ExportFormat.HtmlLight)
{
// Need to escape each path segment while keeping the directory separators intact
return relativeFilePath
@@ -93,7 +93,7 @@ namespace DiscordChatExporter.Core.Exporting
// Try to catch only exceptions related to failed HTTP requests
// https://github.com/Tyrrrz/DiscordChatExporter/issues/332
// https://github.com/Tyrrrz/DiscordChatExporter/issues/372
catch (Exception ex) when (ex is HttpRequestException || ex is OperationCanceledException)
catch (Exception ex) when (ex is HttpRequestException or OperationCanceledException)
{
// TODO: add logging so we can be more liberal with catching exceptions
// We don't want this to crash the exporting process in case of failure
@@ -4,6 +4,7 @@ using System.Text;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exporting.Filtering;
using DiscordChatExporter.Core.Exporting.Partitioning;
using DiscordChatExporter.Core.Utils;
@@ -31,6 +32,8 @@ namespace DiscordChatExporter.Core.Exporting
public PartitionLimit PartitionLimit { get; }
public MessageFilter MessageFilter { get; }
public bool ShouldDownloadMedia { get; }
public bool ShouldReuseMedia { get; }
@@ -45,6 +48,7 @@ namespace DiscordChatExporter.Core.Exporting
Snowflake? after,
Snowflake? before,
PartitionLimit partitionLimit,
MessageFilter messageFilter,
bool shouldDownloadMedia,
bool shouldReuseMedia,
string dateFormat)
@@ -56,6 +60,7 @@ namespace DiscordChatExporter.Core.Exporting
After = after;
Before = before;
PartitionLimit = partitionLimit;
MessageFilter = messageFilter;
ShouldDownloadMedia = shouldDownloadMedia;
ShouldReuseMedia = shouldReuseMedia;
DateFormat = dateFormat;
@@ -0,0 +1,8 @@
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal enum BinaryExpressionKind
{
Or,
And
}
}
@@ -0,0 +1,26 @@
using System;
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal class BinaryExpressionMessageFilter : MessageFilter
{
private readonly MessageFilter _first;
private readonly MessageFilter _second;
private readonly BinaryExpressionKind _kind;
public BinaryExpressionMessageFilter(MessageFilter first, MessageFilter second, BinaryExpressionKind kind)
{
_first = first;
_second = second;
_kind = kind;
}
public override bool Filter(Message message) => _kind switch
{
BinaryExpressionKind.Or => _first.Filter(message) || _second.Filter(message),
BinaryExpressionKind.And => _first.Filter(message) && _second.Filter(message),
_ => throw new InvalidOperationException($"Unknown binary expression kind '{_kind}'.")
};
}
}
@@ -0,0 +1,18 @@
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal class ContainsMessageFilter : MessageFilter
{
private readonly string _text;
public ContainsMessageFilter(string text) => _text = text;
public override bool Filter(Message message) => Regex.IsMatch(
message.Content,
"\\b" + _text + "\\b",
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant
);
}
}
@@ -0,0 +1,17 @@
using System;
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal class FromMessageFilter : MessageFilter
{
private readonly string _value;
public FromMessageFilter(string value) => _value = value;
public override bool Filter(Message message) =>
string.Equals(_value, message.Author.Name, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, message.Author.FullName, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, message.Author.Id.ToString(), StringComparison.OrdinalIgnoreCase);
}
}
@@ -0,0 +1,25 @@
using System;
using System.Linq;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal class HasMessageFilter : MessageFilter
{
private readonly MessageContentMatchKind _kind;
public HasMessageFilter(MessageContentMatchKind kind) => _kind = kind;
public override bool Filter(Message message) => _kind switch
{
MessageContentMatchKind.Link => Regex.IsMatch(message.Content, "https?://\\S*[^\\.,:;\"\'\\s]"),
MessageContentMatchKind.Embed => message.Embeds.Any(),
MessageContentMatchKind.File => message.Attachments.Any(),
MessageContentMatchKind.Video => message.Attachments.Any(file => file.IsVideo),
MessageContentMatchKind.Image => message.Attachments.Any(file => file.IsImage),
MessageContentMatchKind.Sound => message.Attachments.Any(file => file.IsAudio),
_ => throw new InvalidOperationException($"Unknown message content match kind '{_kind}'.")
};
}
}
@@ -0,0 +1,19 @@
using System;
using System.Linq;
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal class MentionsMessageFilter : MessageFilter
{
private readonly string _value;
public MentionsMessageFilter(string value) => _value = value;
public override bool Filter(Message message) => message.MentionedUsers.Any(user =>
string.Equals(_value, user.Name, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, user.FullName, StringComparison.OrdinalIgnoreCase) ||
string.Equals(_value, user.Id.ToString(), StringComparison.OrdinalIgnoreCase)
);
}
}
@@ -0,0 +1,12 @@
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal enum MessageContentMatchKind
{
Link,
Embed,
File,
Video,
Image,
Sound
}
}
@@ -0,0 +1,18 @@
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exporting.Filtering.Parsing;
using Superpower;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
public abstract partial class MessageFilter
{
public abstract bool Filter(Message message);
}
public partial class MessageFilter
{
public static MessageFilter Null { get; } = new NullMessageFilter();
public static MessageFilter Parse(string value) => FilterGrammar.Filter.Parse(value);
}
}
@@ -0,0 +1,13 @@
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal class NegatedMessageFilter : MessageFilter
{
private readonly MessageFilter _filter;
public NegatedMessageFilter(MessageFilter filter) => _filter = filter;
public override bool Filter(Message message) => !_filter.Filter(message);
}
}
@@ -0,0 +1,9 @@
using DiscordChatExporter.Core.Discord.Data;
namespace DiscordChatExporter.Core.Exporting.Filtering
{
internal class NullMessageFilter : MessageFilter
{
public override bool Filter(Message message) => true;
}
}
@@ -0,0 +1,106 @@
using System.Linq;
using DiscordChatExporter.Core.Utils.Extensions;
using Superpower;
using Superpower.Parsers;
namespace DiscordChatExporter.Core.Exporting.Filtering.Parsing
{
internal static class FilterGrammar
{
// Choice(a, b) looks cleaner than a.Or(b)
private static TextParser<T> Choice<T>(params TextParser<T>[] parsers) =>
parsers.Aggregate((current, next) => current.Or(next));
private static readonly TextParser<char> EscapedCharacter =
Character.EqualTo('\\').IgnoreThen(Character.AnyChar);
private static readonly TextParser<string> QuotedString =
from open in Character.In('"', '\'')
from value in Choice(EscapedCharacter, Character.Except(open)).Many().Text()
from close in Character.EqualTo(open)
select value;
private static readonly TextParser<char> FreeCharacter =
Character.Matching(c =>
!char.IsWhiteSpace(c) &&
// Avoid all special tokens used by the grammar
c is not ('(' or ')' or '"' or '\'' or '-' or '|' or '&'),
"any character except whitespace or `(`, `)`, `\"`, `'`, `-`, `|`, `&`"
);
private static readonly TextParser<string> UnquotedString =
Choice(EscapedCharacter, FreeCharacter).AtLeastOnce().Text();
private static readonly TextParser<string> String =
Choice(QuotedString, UnquotedString).Named("text string");
private static readonly TextParser<MessageFilter> ContainsFilter =
String.Select(v => (MessageFilter) new ContainsMessageFilter(v));
private static readonly TextParser<MessageFilter> FromFilter = Span
.EqualToIgnoreCase("from:")
.IgnoreThen(String)
.Select(v => (MessageFilter) new FromMessageFilter(v))
.Named("from:<value>");
private static readonly TextParser<MessageFilter> MentionsFilter = Span
.EqualToIgnoreCase("mentions:")
.IgnoreThen(String)
.Select(v => (MessageFilter) new MentionsMessageFilter(v))
.Named("mentions:<value>");
private static readonly TextParser<MessageFilter> HasFilter = Span
.EqualToIgnoreCase("has:")
.IgnoreThen(Choice(
Span.EqualToIgnoreCase("link").IgnoreThen(Parse.Return(MessageContentMatchKind.Link)),
Span.EqualToIgnoreCase("embed").IgnoreThen(Parse.Return(MessageContentMatchKind.Embed)),
Span.EqualToIgnoreCase("video").IgnoreThen(Parse.Return(MessageContentMatchKind.Video)),
Span.EqualToIgnoreCase("image").IgnoreThen(Parse.Return(MessageContentMatchKind.Image)),
Span.EqualToIgnoreCase("sound").IgnoreThen(Parse.Return(MessageContentMatchKind.Sound))
))
.Select(k => (MessageFilter) new HasMessageFilter(k))
.Named("has:<value>");
private static readonly TextParser<MessageFilter> NegatedFilter = Character
.EqualTo('-')
.IgnoreThen(Parse.Ref(() => StandaloneFilter))
.Select(f => (MessageFilter) new NegatedMessageFilter(f));
private static readonly TextParser<MessageFilter> GroupedFilter =
from open in Character.EqualTo('(')
from content in Parse.Ref(() => BinaryExpressionFilter).Token()
from close in Character.EqualTo(')')
select content;
private static readonly TextParser<MessageFilter> StandaloneFilter = Choice(
GroupedFilter,
FromFilter,
MentionsFilter,
HasFilter,
ContainsFilter
);
private static readonly TextParser<MessageFilter> UnaryExpressionFilter = Choice(
NegatedFilter,
StandaloneFilter
);
private static readonly TextParser<MessageFilter> BinaryExpressionFilter = Parse.Chain(
Choice(
// Explicit operator
Character.In('|', '&').Token().Try(),
// Implicit operator (resolves to 'and')
Character.WhiteSpace.AtLeastOnce().IgnoreThen(Parse.Return(' '))
),
UnaryExpressionFilter,
(op, left, right) => op switch
{
'|' => new BinaryExpressionMessageFilter(left, right, BinaryExpressionKind.Or),
_ => new BinaryExpressionMessageFilter(left, right, BinaryExpressionKind.And)
}
);
public static readonly TextParser<MessageFilter> Filter =
BinaryExpressionFilter.Token().AtEnd();
}
}
@@ -45,7 +45,7 @@ namespace DiscordChatExporter.Core.Exporting
Directory.CreateDirectory(_workingDirPath);
// This catches IOExceptions which is dangerous as we're working also with files
// This retries on IOExceptions which is dangerous as we're also working with files
await Http.ExceptionPolicy.ExecuteAsync(async () =>
{
// Download the file
@@ -56,17 +56,27 @@ namespace DiscordChatExporter.Core.Exporting
}
// Try to set the file date according to the last-modified header
var lastModified = response.Content.Headers.TryGetValue("Last-Modified")?.Pipe(s =>
DateTimeOffset.TryParse(s, CultureInfo.InvariantCulture, DateTimeStyles.None, out var date)
? date
: (DateTimeOffset?) null
);
if (lastModified is not null)
try
{
File.SetCreationTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastWriteTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastAccessTimeUtc(filePath, lastModified.Value.UtcDateTime);
var lastModified = response.Content.Headers.TryGetValue("Last-Modified")?.Pipe(s =>
DateTimeOffset.TryParse(s, CultureInfo.InvariantCulture, DateTimeStyles.None, out var date)
? date
: (DateTimeOffset?) null
);
if (lastModified is not null)
{
File.SetCreationTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastWriteTimeUtc(filePath, lastModified.Value.UtcDateTime);
File.SetLastAccessTimeUtc(filePath, lastModified.Value.UtcDateTime);
}
}
catch
{
// This can apparently fail for some reason.
// https://github.com/Tyrrrz/DiscordChatExporter/issues/585
// Updating file dates is not a critical task, so we'll just
// ignore exceptions thrown here.
}
});
@@ -102,4 +112,4 @@ namespace DiscordChatExporter.Core.Exporting
return PathEx.EscapePath(fileNameWithoutExtension.Truncate(42) + '-' + urlHash + fileExtension);
}
}
}
}
@@ -1,6 +1,6 @@
namespace DiscordChatExporter.Core.Exporting.Partitioning
{
public class FileSizePartitionLimit : PartitionLimit
internal class FileSizePartitionLimit : PartitionLimit
{
private readonly long _limit;
@@ -1,6 +1,6 @@
namespace DiscordChatExporter.Core.Exporting.Partitioning
{
public class MessageCountPartitionLimit : PartitionLimit
internal class MessageCountPartitionLimit : PartitionLimit
{
private readonly long _limit;
@@ -1,9 +1,7 @@
namespace DiscordChatExporter.Core.Exporting.Partitioning
{
public class NullPartitionLimit : PartitionLimit
internal class NullPartitionLimit : PartitionLimit
{
public static NullPartitionLimit Instance { get; } = new();
public override bool IsReached(long messagesWritten, long bytesWritten) => false;
}
}
@@ -11,6 +11,8 @@ namespace DiscordChatExporter.Core.Exporting.Partitioning
public partial class PartitionLimit
{
public static PartitionLimit Null { get; } = new NullPartitionLimit();
private static long? TryParseFileSizeBytes(string value, IFormatProvider? formatProvider = null)
{
var match = Regex.Match(value, @"^\s*(\d+[\.,]?\d*)\s*(\w)?b\s*$", RegexOptions.IgnoreCase);
@@ -1,525 +0,0 @@
/* General */
@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-300.woff);
font-weight: 300;
}
@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-400.woff);
font-weight: 400;
}
@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-500.woff);
font-weight: 500;
}
@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-600.woff);
font-weight: 600;
}
@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-700.woff);
font-weight: 700;
}
body {
font-family: "Whitney", "Helvetica Neue", Helvetica, Arial, sans-serif;
font-size: 17px;
}
a {
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
img {
object-fit: contain;
}
.markdown {
max-width: 100%;
line-height: 1.3;
overflow-wrap: break-word;
}
.preserve-whitespace {
white-space: pre-wrap;
}
.spoiler {
/* width: fit-content; */
display: inline-block;
/* This is more consistent across browsers, the old attribute worked well under Chrome but not FireFox. */
}
.spoiler--hidden {
cursor: pointer;
}
.spoiler-text {
border-radius: 3px;
}
.spoiler--hidden .spoiler-text {
color: rgba(0, 0, 0, 0);
}
.spoiler--hidden .spoiler-text::selection {
color: rgba(0, 0, 0, 0);
}
.spoiler-image {
position: relative;
overflow: hidden;
border-radius: 3px;
}
.spoiler--hidden .spoiler-image {
box-shadow: 0 0 1px 1px rgba(0, 0, 0, 0.1);
}
.spoiler--hidden .spoiler-image * {
filter: blur(44px);
}
.spoiler--hidden .spoiler-image:after {
content: "SPOILER";
color: #dcddde;
background-color: rgba(0, 0, 0, 0.6);
position: absolute;
left: 50%;
top: 50%;
transform: translate(-50%, -50%);
font-weight: 600;
padding: 100%;
border-radius: 20px;
letter-spacing: 0.05em;
font-size: 0.9em;
}
.spoiler--hidden:hover .spoiler-image:after {
color: #fff;
background-color: rgba(0, 0, 0, 0.9);
}
.quote {
margin: 0.1em 0;
padding-left: 0.6em;
border-left: 4px solid;
border-radius: 3px;
}
.pre {
font-family: "Consolas", "Courier New", Courier, monospace;
}
.pre--multiline {
margin-top: 0.25em;
padding: 0.5em;
border: 2px solid;
border-radius: 5px;
}
.pre--inline {
padding: 2px;
border-radius: 3px;
font-size: 0.85em;
}
.mention {
border-radius: 3px;
padding: 0 2px;
color: #7289da;
background: rgba(114, 137, 218, .1);
font-weight: 500;
}
.emoji {
width: 1.25em;
height: 1.25em;
margin: 0 0.06em;
vertical-align: -0.4em;
}
.emoji--small {
width: 1em;
height: 1em;
}
.emoji--large {
width: 2.8em;
height: 2.8em;
}
/* Preamble */
.preamble {
display: grid;
margin: 0 0.3em 0.6em 0.3em;
max-width: 100%;
grid-template-columns: auto 1fr;
}
.preamble__guild-icon-container {
grid-column: 1;
}
.preamble__guild-icon {
max-width: 88px;
max-height: 88px;
}
.preamble__entries-container {
grid-column: 2;
margin-left: 0.6em;
}
.preamble__entry {
font-size: 1.4em;
}
.preamble__entry--small {
font-size: 1em;
}
/* Chatlog */
.chatlog {
max-width: 100%;
}
.chatlog__message-group {
display: grid;
margin: 0 0.6em;
padding: 0.9em 0;
border-top: 1px solid;
grid-template-columns: auto 1fr;
}
.chatlog__reference-symbol {
grid-column: 1;
border-style: solid;
border-width: 2px 0 0 2px;
border-radius: 8px 0 0 0;
margin-left: 16px;
margin-top: 8px;
}
.chatlog__reference {
display: flex;
grid-column: 2;
margin-left: 1.2em;
margin-bottom: 0.25em;
font-size: 0.875em;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
align-items: center;
}
.chatlog__reference-avatar {
border-radius: 50%;
height: 16px;
width: 16px;
margin-right: 0.25em;
}
.chatlog__reference-name {
margin-right: 0.25em;
font-weight: 600;
}
.chatlog__reference-link {
flex-grow: 1;
overflow: hidden;
text-overflow: ellipsis;
}
.chatlog__reference-link:hover {
text-decoration: none;
}
.chatlog__reference-content > * {
display: inline;
}
.chatlog__reference-edited-timestamp {
margin-left: 0.25em;
font-size: 0.8em;
}
.chatlog__author-avatar-container {
grid-column: 1;
width: 40px;
height: 40px;
}
.chatlog__author-avatar {
border-radius: 50%;
height: 40px;
width: 40px;
}
.chatlog__messages {
grid-column: 2;
margin-left: 1.2em;
min-width: 50%;
}
.chatlog__author-name {
font-weight: 500;
}
.chatlog__timestamp {
margin-left: 0.3em;
font-size: 0.75em;
}
.chatlog__message {
padding: 0.1em 0.3em;
margin: 0 -0.3em;
background-color: transparent;
transition: background-color 1s ease;
}
.chatlog__content {
font-size: 0.95em;
word-wrap: break-word;
}
.chatlog__edited-timestamp {
margin-left: 0.15em;
font-size: 0.8em;
}
.chatlog__attachment {
margin-top: 0.3em;
}
.chatlog__attachment-thumbnail {
vertical-align: top;
max-width: 45vw;
max-height: 500px;
border-radius: 3px;
}
.chatlog__attachment-container {
height: 40px;
width: 100%;
max-width: 520px;
padding: 10px;
border: 1px solid;
border-radius: 3px;
overflow: hidden;
}
.chatlog__attachment-icon {
float: left;
height: 100%;
margin-right: 10px;
}
.chatlog__attachment-icon > .a {
fill: #f4f5fb;
d: path("M50,935a25,25,0,0,1-25-25V50A25,25,0,0,1,50,25H519.6L695,201.32V910a25,25,0,0,1-25,25Z");
}
.chatlog__attachment-icon > .b {
fill: #7789c4;
d: path("M509.21,50,670,211.63V910H50V50H509.21M530,0H50A50,50,0,0,0,0,50V910a50,50,0,0,0,50,50H670a50,50,0,0,0,50-50h0V191Z");
}
.chatlog__attachment-icon > .c {
fill: #f4f5fb;
d: path("M530,215a25,25,0,0,1-25-25V50a25,25,0,0,1,16.23-23.41L693.41,198.77A25,25,0,0,1,670,215Z");
}
.chatlog__attachment-icon > .d {
fill: #7789c4;
d: path("M530,70.71,649.29,190H530V70.71M530,0a50,50,0,0,0-50,50V190a50,50,0,0,0,50,50H670a50,50,0,0,0,50-50Z");
}
.chatlog__attachment-filesize {
color: #72767d;
font-size: 12px;
}
.chatlog__attachment-filename {
overflow: hidden;
white-space: nowrap;
text-overflow: ellipsis;
}
.chatlog__embed {
display: flex;
margin-top: 0.3em;
max-width: 520px;
}
.chatlog__embed-color-pill {
flex-shrink: 0;
width: 0.25em;
border-top-left-radius: 3px;
border-bottom-left-radius: 3px;
}
.chatlog__embed-content-container {
display: flex;
flex-direction: column;
padding: 0.5em 0.6em;
border: 1px solid;
border-top-right-radius: 3px;
border-bottom-right-radius: 3px;
}
.chatlog__embed-content {
display: flex;
width: 100%;
}
.chatlog__embed-text {
flex: 1;
}
.chatlog__embed-author {
display: flex;
margin-bottom: 0.3em;
align-items: center;
}
.chatlog__embed-author-icon {
margin-right: 0.5em;
width: 20px;
height: 20px;
border-radius: 50%;
}
.chatlog__embed-author-name {
font-size: 0.875em;
font-weight: 600;
}
.chatlog__embed-title {
margin-bottom: 0.2em;
font-size: 0.875em;
font-weight: 600;
}
.chatlog__embed-description {
font-weight: 500;
font-size: 0.85em;
}
.chatlog__embed-fields {
display: flex;
flex-wrap: wrap;
}
.chatlog__embed-field {
flex: 0;
min-width: 100%;
max-width: 506px;
padding-top: 0.6em;
font-size: 0.875em;
}
.chatlog__embed-field--inline {
flex: 1;
flex-basis: auto;
min-width: 150px;
}
.chatlog__embed-field-name {
margin-bottom: 0.2em;
font-weight: 600;
}
.chatlog__embed-field-value {
font-weight: 500;
}
.chatlog__embed-thumbnail {
flex: 0;
margin-left: 1.2em;
max-width: 80px;
max-height: 80px;
border-radius: 3px;
}
.chatlog__embed-image-container {
margin-top: 0.6em;
}
.chatlog__embed-image {
max-width: 500px;
max-height: 400px;
border-radius: 3px;
}
.chatlog__embed-footer {
margin-top: 0.6em;
}
.chatlog__embed-footer-icon {
margin-right: 0.2em;
width: 20px;
height: 20px;
border-radius: 50%;
vertical-align: middle;
}
.chatlog__embed-footer-text {
font-size: 0.75em;
font-weight: 500;
}
.chatlog__reactions {
display: flex;
}
.chatlog__reaction {
display: flex;
align-items: center;
margin: 0.35em 0.1em 0.1em 0.1em;
padding: 0.2em 0.35em;
border-radius: 3px;
}
.chatlog__reaction-count {
min-width: 9px;
margin-left: 0.35em;
font-size: 0.875em;
}
.chatlog__bot-tag {
position: relative;
top: -.2em;
margin-left: 0.3em;
padding: 0.05em 0.3em;
border-radius: 3px;
vertical-align: middle;
line-height: 1.3;
background: #7289da;
color: #ffffff;
font-size: 0.625em;
font-weight: 500;
}
/* Postamble */
.postamble {
margin: 1.4em 0.3em 0.6em 0.3em;
padding: 1em;
border-top: 1px solid;
}
@@ -1,147 +0,0 @@
/* General */
body {
background-color: #36393e;
color: #dcddde;
}
a {
color: #0096cf;
}
.spoiler-text {
background-color: rgba(255, 255, 255, 0.1);
}
.spoiler--hidden .spoiler-text {
background-color: #202225;
}
.spoiler--hidden:hover .spoiler-text {
background-color: rgba(32, 34, 37, 0.8);
}
.quote {
border-color: #4f545c;
}
.pre {
background-color: #2f3136 !important;
}
.pre--multiline {
border-color: #282b30 !important;
color: #b9bbbe !important;
}
/* === Preamble === */
.preamble__entry {
color: #ffffff;
}
/* Chatlog */
.chatlog__message-group {
border-color: rgba(255, 255, 255, 0.1);
}
.chatlog__reference-symbol {
border-color: #4f545c;
}
.chatlog__reference {
color: #b5b6b8;
}
.chatlog__reference-link {
color: #b5b6b8;
}
.chatlog__reference-link:hover {
color: #ffffff;
}
.chatlog__reference-edited-timestamp {
color: rgba(255, 255, 255, 0.2);
}
.chatlog__author-name {
color: #ffffff;
}
.chatlog__timestamp {
color: rgba(255, 255, 255, 0.2);
}
.chatlog__message--highlighted {
background-color: rgba(114, 137, 218, 0.2) !important;
}
.chatlog__message--pinned {
background-color: rgba(249, 168, 37, 0.05);
}
.chatlog__attachment-container {
background-color: #2f3136;
border-color: #292b2f;
}
.chatlog__edited-timestamp {
color: rgba(255, 255, 255, 0.2);
}
.chatlog__embed-color-pill--default {
background-color: rgba(79, 84, 92, 1);
}
.chatlog__embed-content-container {
background-color: rgba(46, 48, 54, 0.3);
border-color: rgba(46, 48, 54, 0.6);
}
.chatlog__embed-author-name {
color: #ffffff;
}
.chatlog__embed-author-name-link {
color: #ffffff;
}
.chatlog__embed-title {
color: #ffffff;
}
.chatlog__embed-description {
color: rgba(255, 255, 255, 0.6);
}
.chatlog__embed-field-name {
color: #ffffff;
}
.chatlog__embed-field-value {
color: rgba(255, 255, 255, 0.6);
}
.chatlog__embed-footer {
color: rgba(255, 255, 255, 0.6);
}
.chatlog__reaction {
background-color: rgba(255, 255, 255, 0.05);
}
.chatlog__reaction-count {
color: rgba(255, 255, 255, 0.3);
}
/* Postamble */
.postamble {
border-color: rgba(255, 255, 255, 0.1);
}
.postamble__entry {
color: #ffffff;
}
@@ -1,149 +0,0 @@
/* General */
body {
background-color: #ffffff;
color: #23262a;
font-weight: 500;
}
a {
color: #00b0f4;
}
.spoiler-text {
background-color: rgba(0, 0, 0, 0.1);
}
.spoiler--hidden .spoiler-text {
background-color: #b9bbbe;
}
.spoiler--hidden:hover .spoiler-text {
background-color: rgba(185, 187, 190, 0.8);
}
.quote {
border-color: #c7ccd1;
}
.pre {
background-color: #f9f9f9 !important;
}
.pre--multiline {
border-color: #f3f3f3 !important;
color: #657b83 !important;
}
/* Preamble */
.preamble__entry {
color: #2f3136;
}
/* Chatlog */
.chatlog__message-group {
border-color: #eceeef;
}
.chatlog__reference-symbol {
border-color: #c7ccd1;
}
.chatlog__reference {
color: #5f5f60;
}
.chatlog__reference-link {
color: #5f5f60;
}
.chatlog__reference-link:hover {
color: #2f3136;
}
.chatlog__reference-edited-timestamp {
color: #747f8d;
}
.chatlog__author-name {
font-weight: 600;
color: #2f3136;
}
.chatlog__timestamp {
color: #747f8d;
}
.chatlog__message--highlighted {
background-color: rgba(114, 137, 218, 0.2) !important;
}
.chatlog__message--pinned {
background-color: rgba(249, 168, 37, 0.05);
}
.chatlog__attachment-container {
background-color: #f2f3f5;
border-color: #ebedef;
}
.chatlog__edited-timestamp {
color: #747f8d;
}
.chatlog__embed-color-pill--default {
background-color: rgba(227, 229, 232, 1);
}
.chatlog__embed-content-container {
background-color: rgba(249, 249, 249, 0.3);
border-color: rgba(204, 204, 204, 0.3);
}
.chatlog__embed-author-name {
color: #4f545c;
}
.chatlog__embed-author-name-link {
color: #4f545c;
}
.chatlog__embed-title {
color: #4f545c;
}
.chatlog__embed-description {
color: #737f8d;
}
.chatlog__embed-field-name {
color: #36393e;
}
.chatlog__embed-field-value {
color: #737f8d;
}
.chatlog__embed-footer {
color: rgba(79, 83, 91, 0.6);
}
.chatlog__reaction {
background-color: rgba(79, 84, 92, 0.06);
}
.chatlog__reaction-count {
color: #747f8d;
}
/* Postamble */
.postamble {
border-color: #eceeef;
}
.postamble__entry {
color: #2f3136;
}
@@ -36,9 +36,13 @@ namespace DiscordChatExporter.Core.Exporting.Writers.Html
internal partial class MessageGroup
{
public static bool CanJoin(Message message1, Message message2) =>
// Must be from the same author
message1.Author.Id == message2.Author.Id &&
// Author's name must not have changed between messages
string.Equals(message1.Author.FullName, message2.Author.FullName, StringComparison.Ordinal) &&
// Duration between messages must be 7 minutes or less
(message2.Timestamp - message1.Timestamp).Duration().TotalMinutes <= 7 &&
// Other message must not be a reply
message2.Reference is null;
public static MessageGroup Join(IReadOnlyList<Message> messages)
@@ -1,8 +1,10 @@
@using System
@using System.Linq
@using System.Threading.Tasks
@using DiscordChatExporter.Core.Exporting.Writers.Html;
@namespace DiscordChatExporter.Core.Exporting.Writers.Html
@inherits MiniRazor.TemplateBase<DiscordChatExporter.Core.Exporting.Writers.Html.MessageGroupTemplateContext>
@inherits MiniRazor.TemplateBase<MessageGroupTemplateContext>
@{
string FormatDate(DateTimeOffset date) => Model.ExportContext.FormatDate(date);
@@ -45,20 +47,24 @@
}
<div class="chatlog__message-group">
@{/* Referenced message */}
@if (Model.MessageGroup.Reference is not null)
{
<div class="chatlog__reference-symbol">
</div>
<div class="chatlog__reference-symbol"></div>
<div class="chatlog__reference">
@if (Model.MessageGroup.ReferencedMessage is not null)
{
<img class="chatlog__reference-avatar" src="@await ResolveUrlAsync(Model.MessageGroup.ReferencedMessage.Author.AvatarUrl)" alt="Avatar">
<img class="chatlog__reference-avatar" src="@await ResolveUrlAsync(Model.MessageGroup.ReferencedMessage.Author.AvatarUrl)" alt="Avatar" loading="lazy">
<span class="chatlog__reference-name" title="@Model.MessageGroup.ReferencedMessage.Author.FullName" style="@referencedUserColorStyle">@referencedUserNick</span>
<a class="chatlog__reference-link" href="#" onclick="scrollToMessage(event, '@Model.MessageGroup.ReferencedMessage.Id')">
<span class="chatlog__reference-content">
<div class="chatlog__reference-content">
<span class="chatlog__reference-link" onclick="scrollToMessage(event, '@Model.MessageGroup.ReferencedMessage.Id')">
@if (!string.IsNullOrWhiteSpace(Model.MessageGroup.ReferencedMessage.Content))
{
@Raw(FormatMarkdown(Model.MessageGroup.ReferencedMessage.Content))
@Raw(FormatEmbedMarkdown(Model.MessageGroup.ReferencedMessage.Content))
}
else if (Model.MessageGroup.ReferencedMessage.Attachments.Any() || Model.MessageGroup.ReferencedMessage.Embeds.Any())
{
<em>Click to see attachment</em> <span>🖼️</span>
}
else
{
@@ -70,7 +76,7 @@
{
<span class="chatlog__reference-edited-timestamp" title="@FormatDate(Model.MessageGroup.ReferencedMessage.EditedTimestamp.Value)">(edited)</span>
}
</a>
</div>
}
else
{
@@ -80,30 +86,37 @@
}
</div>
}
@{/* Avatar */}
<div class="chatlog__author-avatar-container">
<img class="chatlog__author-avatar" src="@await ResolveUrlAsync(Model.MessageGroup.Author.AvatarUrl)" alt="Avatar">
<img class="chatlog__author-avatar" src="@await ResolveUrlAsync(Model.MessageGroup.Author.AvatarUrl)" alt="Avatar" loading="lazy">
</div>
<div class="chatlog__messages">
@{/* Author name */}
<span class="chatlog__author-name" title="@Model.MessageGroup.Author.FullName" data-user-id="@Model.MessageGroup.Author.Id" style="@userColorStyle">@userNick</span>
@{/* Bot tag */}
@if (Model.MessageGroup.Author.IsBot)
{
<span class="chatlog__bot-tag">BOT</span>
}
@{/* Message timestamp */}
<span class="chatlog__timestamp">@FormatDate(Model.MessageGroup.Timestamp)</span>
@{/* Messages in a group */}
@foreach (var message in Model.MessageGroup.Messages)
{
var isPinnedStyle = message.IsPinned ? "chatlog__message--pinned" : null;
<div class="chatlog__message @isPinnedStyle" data-message-id="@message.Id" id="message-@message.Id">
<div class="chatlog__message @(message.IsPinned ? "chatlog__message--pinned" : null)" data-message-id="@message.Id" id="message-@message.Id" title="Message sent: @FormatDate(message.Timestamp)">
@if (!string.IsNullOrWhiteSpace(message.Content) || message.EditedTimestamp is not null)
{
<div class="chatlog__content">
<div class="markdown">
@{/* Message content */}
<span class="preserve-whitespace">@Raw(FormatMarkdown(message.Content))</span>
@{/* Edit timestamp */}
@if (message.EditedTimestamp is not null)
{
<span class="chatlog__edited-timestamp" title="@FormatDate(message.EditedTimestamp.Value)">(edited)</span>
@@ -112,60 +125,64 @@
</div>
}
@{/* Attachments */}
@foreach (var attachment in message.Attachments)
{
<div class="chatlog__attachment">
<div class="@(attachment.IsSpoiler ? "spoiler spoiler--hidden" : "")" onclick="@(attachment.IsSpoiler ? "showSpoiler(event, this)" : "")">
<div class="@(attachment.IsSpoiler ? "spoiler-image" : "")">
@if (attachment.IsImage)
{
<div class="chatlog__attachment @(attachment.IsSpoiler ? "chatlog__attachment--hidden" : "")" onclick="@(attachment.IsSpoiler ? "showSpoiler(event, this)" : "")">
@{/* Spoiler caption */}
@if (attachment.IsSpoiler)
{
<div class="chatlog__attachment-spoiler-caption">SPOILER</div>
}
@{/* Attachment preview */}
@if (attachment.IsImage)
{
<a href="@await ResolveUrlAsync(attachment.Url)">
<img class="chatlog__attachment-media" src="@await ResolveUrlAsync(attachment.Url)" alt="Image attachment" title="@($"Image: {attachment.FileName} ({attachment.FileSize})")" loading="lazy">
</a>
}
else if (attachment.IsVideo)
{
<video class="chatlog__attachment-media" controls>
<source src="@await ResolveUrlAsync(attachment.Url)" alt="Video attachment" title="@($"Video: {attachment.FileName} ({attachment.FileSize})")">
</video>
}
else if (attachment.IsAudio)
{
<audio class="chatlog__attachment-media" controls>
<source src="@await ResolveUrlAsync(attachment.Url)" alt="Audio attachment" title="@($"Audio: {attachment.FileName} ({attachment.FileSize})")">
</audio>
}
else
{
<div class="chatlog__attachment-generic">
<svg class="chatlog__attachment-generic-icon">
<use href="#icon-attachment" />
</svg>
<div class="chatlog__attachment-generic-name">
<a href="@await ResolveUrlAsync(attachment.Url)">
<img class="chatlog__attachment-thumbnail" src="@await ResolveUrlAsync(attachment.Url)" alt="Image attachment" title="@($"Image: {attachment.FileName} ({attachment.FileSize})")">
@attachment.FileName
</a>
}
else if (attachment.IsVideo)
{
<video controls class="chatlog__attachment-thumbnail">
<source src="@await ResolveUrlAsync(attachment.Url)" alt="Video attachment" title="@($"Video: {attachment.FileName} ({attachment.FileSize})")">
</video>
}
else if (attachment.IsAudio)
{
<audio controls class="chatlog__attachment-thumbnail">
<source src="@await ResolveUrlAsync(attachment.Url)" alt="Audio attachment" title="@($"Audio: {attachment.FileName} ({attachment.FileSize})")">
</audio>
}
else
{
<div class="chatlog__attachment-container">
<svg class="chatlog__attachment-icon" viewBox="0 0 720 960">
<path class="a"/>
<path class="b"/>
<path class="c"/>
<path class="d"/>
</svg>
<div class="chatlog__attachment-filename">
<a href="@await ResolveUrlAsync(attachment.Url)">
@attachment.FileName
</a>
</div>
<div class="chatlog__attachment-filesize">
@attachment.FileSize
</div>
</div>
}
</div>
<div class="chatlog__attachment-generic-size">
@attachment.FileSize
</div>
</div>
</div>
}
</div>
}
@{/* Embeds */}
@foreach (var embed in message.Embeds)
{
var youTubeVideo = embed.TryGetYouTubeVideo();
<div class="chatlog__embed">
@{/* Color pill */}
@if (embed.Color is not null)
{
var embedColorStyle = $"background-color: rgba({embed.Color?.R},{embed.Color?.G},{embed.Color?.B},{embed.Color?.A})";
<div class="chatlog__embed-color-pill" style="@embedColorStyle"></div>
<div class="chatlog__embed-color-pill" style="background-color: rgba(@embed.Color?.R,@embed.Color?.G,@embed.Color?.B,@embed.Color?.A)"></div>
}
else
{
@@ -175,12 +192,13 @@
<div class="chatlog__embed-content-container">
<div class="chatlog__embed-content">
<div class="chatlog__embed-text">
@{/* Embed author */}
@if (embed.Author is not null)
{
<div class="chatlog__embed-author">
@if (!string.IsNullOrWhiteSpace(embed.Author.IconUrl))
{
<img class="chatlog__embed-author-icon" src="@await ResolveUrlAsync(embed.Author.IconUrl)" alt="Author icon">
<img class="chatlog__embed-author-icon" src="@await ResolveUrlAsync(embed.Author.IconProxyUrl ?? embed.Author.IconUrl)" alt="Author icon" loading="lazy" onerror="this.style.visibility='hidden'">
}
@if (!string.IsNullOrWhiteSpace(embed.Author.Name))
@@ -199,6 +217,7 @@
</div>
}
@{/* Embed title */}
@if (!string.IsNullOrWhiteSpace(embed.Title))
{
<div class="chatlog__embed-title">
@@ -215,21 +234,27 @@
</div>
}
@if (!string.IsNullOrWhiteSpace(embed.Description))
@{/* Embed description (with special casing for YouTube videos) */}
@if (youTubeVideo is not null)
{
<div class="chatlog__embed-youtube-container">
<iframe class="chatlog__embed-youtube" src="@youTubeVideo.Url" width="400" height="225"></iframe>
</div>
}
else if (!string.IsNullOrWhiteSpace(embed.Description))
{
<div class="chatlog__embed-description">
<div class="markdown preserve-whitespace">@Raw(FormatEmbedMarkdown(embed.Description))</div>
</div>
}
@{/* Embed fields */}
@if (embed.Fields.Any())
{
<div class="chatlog__embed-fields">
@foreach (var field in embed.Fields)
{
var isInlineStyle = field.IsInline ? "chatlog__embed-field--inline" : null;
<div class="chatlog__embed-field @isInlineStyle">
<div class="chatlog__embed-field @(field.IsInline ? "chatlog__embed-field--inline" : null)">
@if (!string.IsNullOrWhiteSpace(field.Name))
{
<div class="chatlog__embed-field-name">
@@ -249,34 +274,39 @@
}
</div>
@if (embed.Thumbnail is not null)
@{/* Embed content (not shown for YouTube videos) */}
@if (embed.Thumbnail is not null && !string.IsNullOrWhiteSpace(embed.Thumbnail.Url) && youTubeVideo is null)
{
<div class="chatlog__embed-thumbnail-container">
<a class="chatlog__embed-thumbnail-link" href="@await ResolveUrlAsync(embed.Thumbnail.Url)">
<img class="chatlog__embed-thumbnail" src="@await ResolveUrlAsync(embed.Thumbnail.Url)" alt="Thumbnail">
<a class="chatlog__embed-thumbnail-link" href="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)">
<img class="chatlog__embed-thumbnail" src="@await ResolveUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url)" alt="Thumbnail" loading="lazy">
</a>
</div>
}
</div>
@if (embed.Image is not null)
@{/* Embed image */}
@if (embed.Image is not null && !string.IsNullOrWhiteSpace(embed.Image.Url))
{
<div class="chatlog__embed-image-container">
<a class="chatlog__embed-image-link" href="@await ResolveUrlAsync(embed.Image.Url)">
<img class="chatlog__embed-image" src="@await ResolveUrlAsync(embed.Image.Url)" alt="Image">
<a class="chatlog__embed-image-link" href="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)">
<img class="chatlog__embed-image" src="@await ResolveUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url)" alt="Image" loading="lazy">
</a>
</div>
}
@{/* Embed footer & icon */}
@if (embed.Footer is not null || embed.Timestamp is not null)
{
<div class="chatlog__embed-footer">
@{/* Footer icon */}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.IconUrl))
{
<img class="chatlog__embed-footer-icon" src="@await ResolveUrlAsync(embed.Footer.IconUrl)" alt="Footer icon">
<img class="chatlog__embed-footer-icon" src="@await ResolveUrlAsync(embed.Footer.IconProxyUrl ?? embed.Footer.IconUrl)" alt="Footer icon" loading="lazy">
}
<span class="chatlog__embed-footer-text">
@{/* Footer text */}
@if (!string.IsNullOrWhiteSpace(embed.Footer?.Text))
{
@embed.Footer.Text
@@ -287,6 +317,7 @@
@(" • ")
}
@{/* Embed timestamp */}
@if (embed.Timestamp is not null)
{
@FormatDate(embed.Timestamp.Value)
@@ -298,13 +329,14 @@
</div>
}
@{/* Message reactions */}
@if (message.Reactions.Any())
{
<div class="chatlog__reactions">
@foreach (var reaction in message.Reactions)
{
<div class="chatlog__reaction">
<img class="emoji emoji--small" alt="@reaction.Emoji.Name" title="@reaction.Emoji.Name" src="@await ResolveUrlAsync(reaction.Emoji.ImageUrl)">
<div class="chatlog__reaction" title="@reaction.Emoji.Code">
<img class="emoji emoji--small" alt="@reaction.Emoji.Name" src="@await ResolveUrlAsync(reaction.Emoji.ImageUrl)" loading="lazy">
<span class="chatlog__reaction-count">@reaction.Count</span>
</div>
}
@@ -1,6 +1,9 @@
@namespace DiscordChatExporter.Core.Exporting.Writers.Html
@inherits MiniRazor.TemplateBase<DiscordChatExporter.Core.Exporting.Writers.Html.PostambleTemplateContext>
@using DiscordChatExporter.Core.Exporting.Writers.Html;
@namespace DiscordChatExporter.Core.Exporting.Writers.Html
@inherits MiniRazor.TemplateBase<PostambleTemplateContext>
@{/* Close elements opened by preamble */}
</div>
<div class="postamble">
@@ -1,15 +1,19 @@
@using System
@using System.Threading.Tasks
@using Tyrrrz.Extensions
@using DiscordChatExporter.Core.Exporting.Writers.Html;
@namespace DiscordChatExporter.Core.Exporting.Writers.Html
@inherits MiniRazor.TemplateBase<DiscordChatExporter.Core.Exporting.Writers.Html.PreambleTemplateContext>
@inherits MiniRazor.TemplateBase<PreambleTemplateContext>
@{
string Themed(string darkVariant, string lightVariant) =>
string.Equals(Model.ThemeName, "Dark", StringComparison.OrdinalIgnoreCase)
? darkVariant
: lightVariant;
string FormatDate(DateTimeOffset date) => Model.ExportContext.FormatDate(date);
ValueTask<string> ResolveUrlAsync(string url) => Model.ExportContext.ResolveMediaUrlAsync(url);
string GetStyleSheet(string name) => Model.GetType().Assembly.GetManifestResourceString($"DiscordChatExporter.Core.Exporting.Writers.Html.{name}.css");
}
<!DOCTYPE html>
@@ -20,13 +24,595 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width">
@{/* Styling */}
<style>
@Raw(GetStyleSheet("Core"))
</style>
<style>
@Raw(GetStyleSheet(Model.ThemeName))
@@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-300.woff);
font-weight: 300;
}
@@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-400.woff);
font-weight: 400;
}
@@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-500.woff);
font-weight: 500;
}
@@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-600.woff);
font-weight: 600;
}
@@font-face {
font-family: Whitney;
src: url(https://cdn.jsdelivr.net/gh/Tyrrrz/DiscordFonts@master/whitney-700.woff);
font-weight: 700;
}
body {
background-color: @Themed("#36393e", "#ffffff");
color: @Themed("#dcddde", "#23262a");
font-family: "Whitney", "Helvetica Neue", Helvetica, Arial, sans-serif;
font-size: 17px;
font-weight: @Themed("400", "500");
}
a {
color: @Themed("#00aff4", "#0068e0");
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
img {
object-fit: contain;
}
.markdown {
max-width: 100%;
line-height: 1.3;
overflow-wrap: break-word;
}
.preserve-whitespace {
white-space: pre-wrap;
}
.spoiler-text {
background-color: @Themed("rgba(255, 255, 255, 0.1)", "rgba(0, 0, 0, 0.1)");
border-radius: 3px;
}
.spoiler-text--hidden {
cursor: pointer;
background-color: @Themed("#202225", "#b9bbbe");
color: rgba(0, 0, 0, 0);
}
.spoiler-text--hidden:hover {
background-color: @Themed("rgba(32, 34, 37, 0.8)", "rgba(185, 187, 190, 0.8)");
}
.spoiler-text--hidden::selection {
color: rgba(0, 0, 0, 0);
}
.quote {
margin: 0.1em 0;
padding-left: 0.6em;
border-left: 4px solid @Themed("#4f545c", "#c7ccd1");
border-radius: 3px;
}
.pre {
background-color: @Themed("#2f3136", "#f9f9f9");
font-family: "Consolas", "Courier New", Courier, monospace;
}
.pre--multiline {
margin-top: 0.25em;
padding: 0.5em;
border: 2px solid @Themed("#282b30", "#f3f3f3");
border-radius: 5px;
color: @Themed("#b9bbbe", "#657b83");
}
@{/* Override Highlight.js styles with a higher specificity selector */}
.pre--multiline.hljs {
background-color: @Themed("#2f3136", "#f9f9f9");
color: @Themed("#b9bbbe", "#657b83");
}
.pre--inline {
padding: 2px;
border-radius: 3px;
font-size: 0.85em;
}
.mention {
border-radius: 3px;
padding: 0 2px;
color: #7289da;
background-color: rgba(114, 137, 218, .1);
font-weight: 500;
}
.timestamp {
border-radius: 3px;
padding: 0 2px;
background-color: @Themed("rgba(255, 255, 255, 0.06)", "rgba(6, 6, 7, 0.08)");
}
.emoji {
width: 1.325em;
height: 1.325em;
margin: 0 0.06em;
vertical-align: -0.4em;
}
.emoji--small {
width: 1em;
height: 1em;
}
.emoji--large {
width: 2.8em;
height: 2.8em;
}
.preamble {
display: grid;
margin: 0 0.3em 0.6em 0.3em;
max-width: 100%;
grid-template-columns: auto 1fr;
}
.preamble__guild-icon-container {
grid-column: 1;
}
.preamble__guild-icon {
max-width: 88px;
max-height: 88px;
}
.preamble__entries-container {
grid-column: 2;
margin-left: 0.6em;
}
.preamble__entry {
font-size: 1.4em;
color: @Themed("#ffffff", "#2f3136");
}
.preamble__entry--small {
font-size: 1em;
}
.chatlog {
max-width: 100%;
}
.chatlog__message-group {
display: grid;
margin: 0 0.6em;
padding: 0.9em 0;
border-top: 1px solid @Themed("rgba(255, 255, 255, 0.1)", "#eceeef");
grid-template-columns: auto 1fr;
}
.chatlog__reference-symbol {
grid-column: 1;
margin: 8px 4px 4px 18px;
border-left: 2px solid @Themed("#4f545c", "#c7ccd1");
border-top: 2px solid @Themed("#4f545c", "#c7ccd1");
border-radius: 8px 0 0 0;
}
.chatlog__reference {
display: flex;
grid-column: 2;
margin-bottom: 0.25em;
font-size: 0.875em;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
align-items: center;
color: @Themed("#b5b6b8", "#5f5f60");
}
.chatlog__reference-avatar {
border-radius: 50%;
width: 16px;
height: 16px;
margin-right: 0.25em;
}
.chatlog__reference-name {
margin-right: 0.25em;
font-weight: 600;
}
.chatlog__reference-content {
overflow: hidden;
text-overflow: ellipsis;
}
.chatlog__reference-content a:hover {
text-decoration: none;
}
.chatlog__reference-link {
cursor: pointer;
color: @Themed("#b5b6b8", "#5f5f60");
}
.chatlog__reference-link * {
display: inline;
pointer-events: none;
}
.chatlog__reference-link:hover {
color: @Themed("#ffffff", "#2f3136");
}
.chatlog__reference-link:hover *:not(.spoiler-text) {
color: @Themed("#ffffff", "#2f3136");
}
.chatlog__reference-edited-timestamp {
margin-left: 0.25em;
font-size: 0.8em;
color: @Themed("rgba(255, 255, 255, 0.2)", "#747f8d");
}
.chatlog__author-avatar-container {
grid-column: 1;
width: 40px;
height: 40px;
margin-right: 16px;
}
.chatlog__author-avatar {
border-radius: 50%;
width: 40px;
height: 40px;
}
.chatlog__messages {
grid-column: 2;
min-width: 50%;
}
.chatlog__author-name {
font-weight: @Themed("500", "600");
color: @Themed("#ffffff", "#2f3136");
}
.chatlog__timestamp {
margin-left: 0.3em;
font-size: 0.75em;
color: @Themed("rgba(255, 255, 255, 0.2)", "#747f8d");
}
.chatlog__message {
padding: 0.1em 0.3em;
margin: 0 -0.3em;
background-color: transparent;
transition: background-color 1s ease;
}
.chatlog__message--highlighted {
background-color: @Themed("rgba(114, 137, 218, 0.2)", "rgba(114, 137, 218, 0.2)");
}
.chatlog__message--pinned {
background-color: @Themed("rgba(249, 168, 37, 0.05)", "rgba(249, 168, 37, 0.05)");
}
.chatlog__content {
font-size: 0.95em;
word-wrap: break-word;
}
.chatlog__edited-timestamp {
margin-left: 0.15em;
font-size: 0.8em;
}
.chatlog__attachment {
display: inline-block;
position: relative;
margin-top: 0.3em;
border-radius: 3px;
overflow: hidden;
}
.chatlog__attachment--hidden {
cursor: pointer;
box-shadow: 0 0 1px 1px rgba(0, 0, 0, 0.1);
}
.chatlog__attachment--hidden * {
pointer-events: none;
}
.chatlog__attachment-spoiler-caption {
display: none;
z-index: 999;
position: absolute;
left: 50%;
top: 50%;
transform: translate(-50%, -50%);
padding: 0.4em 0.8em;
background-color: rgba(0, 0, 0, 0.9);
border-radius: 20px;
color: #dcddde;
font-size: 0.9em;
font-weight: 600;
letter-spacing: 0.05em;
}
.chatlog__attachment--hidden .chatlog__attachment-spoiler-caption {
display: block;
}
.chatlog__attachment--hidden:hover .chatlog__attachment-spoiler-caption {
color: #fff;
}
.chatlog__attachment-media {
vertical-align: top;
max-width: 45vw;
max-height: 500px;
border-radius: 3px;
}
.chatlog__attachment--hidden .chatlog__attachment-media {
filter: blur(44px);
}
.chatlog__attachment-generic {
width: 100%;
max-width: 520px;
height: 40px;
padding: 10px;
background-color: @Themed("#2f3136", "#f2f3f5");
border: 1px solid @Themed("#292b2f", "#ebedef");
border-radius: 3px;
overflow: hidden;
}
.chatlog__attachment--hidden .chatlog__attachment-generic {
filter: blur(44px);
}
.chatlog__attachment-generic-icon {
float: left;
width: 30px;
height: 100%;
margin-right: 10px;
}
.chatlog__attachment-generic-size {
color: #72767d;
font-size: 12px;
}
.chatlog__attachment-generic-name {
overflow: hidden;
white-space: nowrap;
text-overflow: ellipsis;
}
.chatlog__edited-timestamp {
color: @Themed("rgba(255, 255, 255, 0.2)", "#747f8d");
}
.chatlog__embed {
display: flex;
margin-top: 0.3em;
max-width: 520px;
}
.chatlog__embed-color-pill {
flex-shrink: 0;
width: 0.25em;
border-top-left-radius: 3px;
border-bottom-left-radius: 3px;
}
.chatlog__embed-color-pill--default {
background-color: @Themed("#202225", "rgba(227, 229, 232, 1)");
}
.chatlog__embed-content-container {
display: flex;
flex-direction: column;
padding: 0.5em 0.6em;
background-color: @Themed("rgba(46, 48, 54, 0.3)", "rgba(249, 249, 249, 0.3)");
border: 1px solid @Themed("rgba(46, 48, 54, 0.6)", "rgba(204, 204, 204, 0.3)");
border-top-right-radius: 3px;
border-bottom-right-radius: 3px;
}
.chatlog__embed-content {
display: flex;
width: 100%;
}
.chatlog__embed-text {
flex: 1;
}
.chatlog__embed-author {
display: flex;
margin-bottom: 0.3em;
align-items: center;
}
.chatlog__embed-author-icon {
margin-right: 0.5em;
width: 20px;
height: 20px;
border-radius: 50%;
}
.chatlog__embed-author-name {
font-size: 0.875em;
font-weight: 600;
color: @Themed("#ffffff", "#4f545c")
}
.chatlog__embed-author-name-link {
color: @Themed("#ffffff", "#4f545c");
}
.chatlog__embed-title {
margin-bottom: 0.2em;
font-size: 0.875em;
font-weight: 600;
color: @Themed("#ffffff", "#4f545c");
}
.chatlog__embed-description {
font-weight: 500;
font-size: 0.85em;
color: @Themed("#dcddde", "#2e3338");
}
.chatlog__embed-fields {
display: flex;
flex-wrap: wrap;
}
.chatlog__embed-field {
flex: 0;
min-width: 100%;
max-width: 506px;
padding-top: 0.6em;
font-size: 0.875em;
}
.chatlog__embed-field--inline {
flex: 1;
flex-basis: auto;
min-width: 150px;
}
.chatlog__embed-field-name {
margin-bottom: 0.2em;
font-weight: 600;
color: @Themed("#ffffff", "#36393e");
}
.chatlog__embed-field-value {
font-weight: 500;
color: @Themed("#dcddde", "#2e3338");
}
.chatlog__embed-thumbnail {
flex: 0;
margin-left: 1.2em;
max-width: 80px;
max-height: 80px;
border-radius: 3px;
}
.chatlog__embed-image-container {
margin-top: 0.6em;
}
.chatlog__embed-image {
max-width: 500px;
max-height: 400px;
border-radius: 3px;
}
.chatlog__embed-footer {
margin-top: 0.6em;
color: @Themed("#dcddde", "#2e3338");
}
.chatlog__embed-footer-icon {
margin-right: 0.2em;
width: 20px;
height: 20px;
border-radius: 50%;
vertical-align: middle;
}
.chatlog__embed-footer-text {
vertical-align: middle;
font-size: 0.75em;
font-weight: 500;
}
.chatlog__embed-youtube-container {
margin-top: 0.6em;
}
.chatlog__embed-youtube {
border: 0;
border-radius: 3px;
}
.chatlog__reactions {
display: flex;
}
.chatlog__reaction {
display: flex;
align-items: center;
margin: 0.35em 0.1em 0.1em 0.1em;
padding: 0.2em 0.35em;
background-color: @Themed("rgba(255, 255, 255, 0.05)", "rgba(79, 84, 92, 0.06)");
border-radius: 3px;
}
.chatlog__reaction-count {
min-width: 9px;
margin-left: 0.35em;
font-size: 0.875em;
color: @Themed("rgba(255, 255, 255, 0.3)", "#747f8d");
}
.chatlog__bot-tag {
position: relative;
top: -.2em;
margin-left: 0.3em;
padding: 0.05em 0.3em;
border-radius: 3px;
vertical-align: middle;
line-height: 1.3;
background-color: #5865F2;
color: #ffffff;
font-size: 0.625em;
font-weight: 500;
}
.postamble {
margin: 1.4em 0.3em 0.6em 0.3em;
padding: 1em;
border-top: 1px solid @Themed("rgba(255, 255, 255, 0.1)", "#eceeef");
}
.postamble__entry {
color: @Themed("#ffffff", "#2f3136");
}
</style>
@{/* Syntax highlighting */}
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/styles/solarized-@(Model.ThemeName.ToLowerInvariant()).min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.6/highlight.min.js"></script>
<script>
@@ -35,6 +621,7 @@
});
</script>
@{/* Scripts */}
<script>
function scrollToMessage(event, id) {
var element = document.getElementById('message-' + id);
@@ -56,18 +643,32 @@
}
function showSpoiler(event, element) {
if (element && element.classList.contains('spoiler--hidden')) {
if (element && element.classList.contains('spoiler-text--hidden')) {
event.preventDefault();
element.classList.remove('spoiler--hidden');
element.classList.remove('spoiler-text--hidden');
}
if (element && element.classList.contains('chatlog__attachment--hidden')) {
event.preventDefault();
element.classList.remove('chatlog__attachment--hidden');
}
}
</script>
@{/* Icons */}
<svg style="display: none">
<symbol id="icon-attachment" viewBox="0 0 720 960">
<path fill="#f4f5fb" d="M50,935a25,25,0,0,1-25-25V50A25,25,0,0,1,50,25H519.6L695,201.32V910a25,25,0,0,1-25,25Z" />
<path fill="#7789c4" d="M509.21,50,670,211.63V910H50V50H509.21M530,0H50A50,50,0,0,0,0,50V910a50,50,0,0,0,50,50H670a50,50,0,0,0,50-50h0V191Z" />
<path fill="#f4f5fb" d="M530,215a25,25,0,0,1-25-25V50a25,25,0,0,1,16.23-23.41L693.41,198.77A25,25,0,0,1,670,215Z" />
<path fill="#7789c4" d="M530,70.71,649.29,190H530V70.71M530,0a50,50,0,0,0-50,50V190a50,50,0,0,0,50,50H670a50,50,0,0,0,50-50Z" />
</symbol>
</svg>
</head>
<body>
<div class="preamble">
<div class="preamble__guild-icon-container">
<img class="preamble__guild-icon" src="@await ResolveUrlAsync(Model.ExportContext.Request.Guild.IconUrl)" alt="Guild icon">
<img class="preamble__guild-icon" src="@await ResolveUrlAsync(Model.ExportContext.Request.Guild.IconUrl)" alt="Guild icon" loading="lazy">
</div>
<div class="preamble__entries-container">
<div class="preamble__entry">@Model.ExportContext.Request.Guild.Name</div>
@@ -98,4 +699,5 @@
</div>
</div>
@{/* Preamble cuts off at this point */}
<div class="chatlog">
@@ -20,6 +20,8 @@ namespace DiscordChatExporter.Core.Exporting.Writers
{
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
Indented = true,
// Validation errors may mask actual failures
// https://github.com/Tyrrrz/DiscordChatExporter/issues/413
SkipValidation = true
});
}
@@ -48,7 +50,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteString("url", embedAuthor.Url);
if (!string.IsNullOrWhiteSpace(embedAuthor.IconUrl))
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedAuthor.IconUrl));
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedAuthor.IconProxyUrl ?? embedAuthor.IconUrl));
_writer.WriteEndObject();
await _writer.FlushAsync();
@@ -59,7 +61,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteStartObject("thumbnail");
if (!string.IsNullOrWhiteSpace(embedThumbnail.Url))
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedThumbnail.Url));
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedThumbnail.ProxyUrl ?? embedThumbnail.Url));
_writer.WriteNumber("width", embedThumbnail.Width);
_writer.WriteNumber("height", embedThumbnail.Height);
@@ -73,7 +75,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteStartObject("image");
if (!string.IsNullOrWhiteSpace(embedImage.Url))
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedImage.Url));
_writer.WriteString("url", await Context.ResolveMediaUrlAsync(embedImage.ProxyUrl ?? embedImage.Url));
_writer.WriteNumber("width", embedImage.Width);
_writer.WriteNumber("height", embedImage.Height);
@@ -89,7 +91,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers
_writer.WriteString("text", embedFooter.Text);
if (!string.IsNullOrWhiteSpace(embedFooter.IconUrl))
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedFooter.IconUrl));
_writer.WriteString("iconUrl", await Context.ResolveMediaUrlAsync(embedFooter.IconProxyUrl ?? embedFooter.IconUrl));
_writer.WriteEndObject();
await _writer.FlushAsync();
@@ -300,4 +302,4 @@ namespace DiscordChatExporter.Core.Exporting.Writers
await base.DisposeAsync();
}
}
}
}
@@ -6,7 +6,7 @@ using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Markdown.Ast;
using DiscordChatExporter.Core.Markdown.Parsing;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
@@ -39,7 +39,7 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
TextFormatting.Underline => ("<u>", "</u>"),
TextFormatting.Strikethrough => ("<s>", "</s>"),
TextFormatting.Spoiler => (
"<span class=\"spoiler spoiler--hidden\" onclick=\"showSpoiler(event, this)\"><span class=\"spoiler-text\">", "</span></span>"),
"<span class=\"spoiler-text spoiler-text--hidden\" onclick=\"showSpoiler(event, this)\">", "</span>"),
TextFormatting.Quote => ("<div class=\"quote\">", "</div>"),
_ => throw new ArgumentOutOfRangeException(nameof(formatted.Formatting))
};
@@ -75,67 +75,6 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
return base.VisitMultiLineCodeBlock(multiLineCodeBlock);
}
protected override MarkdownNode VisitMention(MentionNode mention)
{
var mentionId = Snowflake.TryParse(mention.Id);
if (mention.Type == MentionType.Meta)
{
_buffer
.Append("<span class=\"mention\">")
.Append("@").Append(HtmlEncode(mention.Id))
.Append("</span>");
}
else if (mention.Type == MentionType.User)
{
var member = mentionId?.Pipe(_context.TryGetMember);
var fullName = member?.User.FullName ?? "Unknown";
var nick = member?.Nick ?? "Unknown";
_buffer
.Append($"<span class=\"mention\" title=\"{HtmlEncode(fullName)}\">")
.Append("@").Append(HtmlEncode(nick))
.Append("</span>");
}
else if (mention.Type == MentionType.Channel)
{
var channel = mentionId?.Pipe(_context.TryGetChannel);
var name = channel?.Name ?? "deleted-channel";
_buffer
.Append("<span class=\"mention\">")
.Append("#").Append(HtmlEncode(name))
.Append("</span>");
}
else if (mention.Type == MentionType.Role)
{
var role = mentionId?.Pipe(_context.TryGetRole);
var name = role?.Name ?? "deleted-role";
var color = role?.Color;
var style = color is not null
? $"color: rgb({color?.R}, {color?.G}, {color?.B}); background-color: rgba({color?.R}, {color?.G}, {color?.B}, 0.1);"
: "";
_buffer
.Append($"<span class=\"mention\" style=\"{style}\">")
.Append("@").Append(HtmlEncode(name))
.Append("</span>");
}
return base.VisitMention(mention);
}
protected override MarkdownNode VisitEmoji(EmojiNode emoji)
{
var emojiImageUrl = Emoji.GetImageUrl(emoji.Id, emoji.Name, emoji.IsAnimated);
var jumboClass = _isJumbo ? "emoji--large" : "";
_buffer
.Append($"<img class=\"emoji {jumboClass}\" alt=\"{emoji.Name}\" title=\"{emoji.Name}\" src=\"{emojiImageUrl}\">");
return base.VisitEmoji(emoji);
}
protected override MarkdownNode VisitLink(LinkNode link)
{
// Extract message ID if the link points to a Discord message
@@ -158,6 +97,78 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
return base.VisitLink(link);
}
protected override MarkdownNode VisitEmoji(EmojiNode emoji)
{
var emojiImageUrl = Emoji.GetImageUrl(emoji.Id, emoji.Name, emoji.IsAnimated);
var jumboClass = _isJumbo ? "emoji--large" : "";
_buffer
.Append($"<img loading=\"lazy\" class=\"emoji {jumboClass}\" alt=\"{emoji.Name}\" title=\"{emoji.Code}\" src=\"{emojiImageUrl}\">");
return base.VisitEmoji(emoji);
}
protected override MarkdownNode VisitMention(MentionNode mention)
{
var mentionId = Snowflake.TryParse(mention.Id);
if (mention.Kind == MentionKind.Meta)
{
_buffer
.Append("<span class=\"mention\">")
.Append("@").Append(HtmlEncode(mention.Id))
.Append("</span>");
}
else if (mention.Kind == MentionKind.User)
{
var member = mentionId?.Pipe(_context.TryGetMember);
var fullName = member?.User.FullName ?? "Unknown";
var nick = member?.Nick ?? "Unknown";
_buffer
.Append($"<span class=\"mention\" title=\"{HtmlEncode(fullName)}\">")
.Append("@").Append(HtmlEncode(nick))
.Append("</span>");
}
else if (mention.Kind == MentionKind.Channel)
{
var channel = mentionId?.Pipe(_context.TryGetChannel);
var symbol = channel?.IsVoiceChannel == true ? "🔊" : "#";
var name = channel?.Name ?? "deleted-channel";
_buffer
.Append("<span class=\"mention\">")
.Append(symbol).Append(HtmlEncode(name))
.Append("</span>");
}
else if (mention.Kind == MentionKind.Role)
{
var role = mentionId?.Pipe(_context.TryGetRole);
var name = role?.Name ?? "deleted-role";
var color = role?.Color;
var style = color is not null
? $"color: rgb({color?.R}, {color?.G}, {color?.B}); background-color: rgba({color?.R}, {color?.G}, {color?.B}, 0.1);"
: "";
_buffer
.Append($"<span class=\"mention\" style=\"{style}\">")
.Append("@").Append(HtmlEncode(name))
.Append("</span>");
}
return base.VisitMention(mention);
}
protected override MarkdownNode VisitUnixTimestamp(UnixTimestampNode timestamp)
{
_buffer
.Append("<span class=\"timestamp\">")
.Append(HtmlEncode(_context.FormatDate(timestamp.Value)))
.Append("</span>");
return base.VisitUnixTimestamp(timestamp);
}
}
internal partial class HtmlMarkdownVisitor
@@ -1,7 +1,7 @@
using System.Text;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Markdown;
using DiscordChatExporter.Core.Markdown.Ast;
using DiscordChatExporter.Core.Markdown.Parsing;
using DiscordChatExporter.Core.Utils.Extensions;
namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
@@ -23,28 +23,43 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
return base.VisitText(text);
}
protected override MarkdownNode VisitEmoji(EmojiNode emoji)
{
_buffer.Append(
emoji.IsCustomEmoji
? $":{emoji.Name}:"
: emoji.Name
);
return base.VisitEmoji(emoji);
}
protected override MarkdownNode VisitMention(MentionNode mention)
{
var mentionId = Snowflake.TryParse(mention.Id);
if (mention.Type == MentionType.Meta)
if (mention.Kind == MentionKind.Meta)
{
_buffer.Append($"@{mention.Id}");
}
else if (mention.Type == MentionType.User)
else if (mention.Kind == MentionKind.User)
{
var member = mentionId?.Pipe(_context.TryGetMember);
var name = member?.User.Name ?? "Unknown";
_buffer.Append($"@{name}");
}
else if (mention.Type == MentionType.Channel)
else if (mention.Kind == MentionKind.Channel)
{
var channel = mentionId?.Pipe(_context.TryGetChannel);
var name = channel?.Name ?? "deleted-channel";
_buffer.Append($"#{name}");
// Voice channel marker
if (channel?.IsVoiceChannel == true)
_buffer.Append(" [voice]");
}
else if (mention.Type == MentionType.Role)
else if (mention.Kind == MentionKind.Role)
{
var role = mentionId?.Pipe(_context.TryGetRole);
var name = role?.Name ?? "deleted-role";
@@ -55,15 +70,13 @@ namespace DiscordChatExporter.Core.Exporting.Writers.MarkdownVisitors
return base.VisitMention(mention);
}
protected override MarkdownNode VisitEmoji(EmojiNode emoji)
protected override MarkdownNode VisitUnixTimestamp(UnixTimestampNode timestamp)
{
_buffer.Append(
emoji.IsCustomEmoji
? $":{emoji.Name}:"
: emoji.Name
_context.FormatDate(timestamp.Value)
);
return base.VisitEmoji(emoji);
return base.VisitUnixTimestamp(timestamp);
}
}
@@ -75,10 +75,10 @@ namespace DiscordChatExporter.Core.Exporting.Writers
}
if (!string.IsNullOrWhiteSpace(embed.Thumbnail?.Url))
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Thumbnail.Url));
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Thumbnail.ProxyUrl ?? embed.Thumbnail.Url));
if (!string.IsNullOrWhiteSpace(embed.Image?.Url))
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Image.Url));
await _writer.WriteLineAsync(await Context.ResolveMediaUrlAsync(embed.Image.ProxyUrl ?? embed.Image.Url));
if (!string.IsNullOrWhiteSpace(embed.Footer?.Text))
await _writer.WriteLineAsync(embed.Footer.Text);
@@ -1,25 +0,0 @@
namespace DiscordChatExporter.Core.Markdown.Ast
{
internal enum MentionType
{
Meta,
User,
Channel,
Role
}
internal class MentionNode : MarkdownNode
{
public string Id { get; }
public MentionType Type { get; }
public MentionNode(string id, MentionType type)
{
Id = id;
Type = type;
}
public override string ToString() => $"<{Type} mention> {Id}";
}
}
@@ -1,11 +1,20 @@
namespace DiscordChatExporter.Core.Markdown.Ast
using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Markdown
{
internal class EmojiNode : MarkdownNode
{
// Only present on custom emoji
public string? Id { get; }
// Name of custom emoji (e.g. LUL) or actual representation of standard emoji (e.g. 🙂)
public string Name { get; }
// Name of custom emoji (e.g. LUL) or name of standard emoji (e.g. slight_smile)
public string Code => !string.IsNullOrWhiteSpace(Id)
? Name
: EmojiIndex.TryGetCode(Name) ?? Name;
public bool IsAnimated { get; }
public bool IsCustomEmoji => !string.IsNullOrWhiteSpace(Id);
@@ -1,17 +1,7 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown.Ast
namespace DiscordChatExporter.Core.Markdown
{
internal enum TextFormatting
{
Bold,
Italic,
Underline,
Strikethrough,
Spoiler,
Quote
}
internal class FormattedNode : MarkdownNode
{
public TextFormatting Formatting { get; }
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown.Ast
namespace DiscordChatExporter.Core.Markdown
{
internal class InlineCodeBlockNode : MarkdownNode
{
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown.Ast
namespace DiscordChatExporter.Core.Markdown
{
internal class LinkNode : MarkdownNode
{
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown.Ast
namespace DiscordChatExporter.Core.Markdown
{
internal abstract class MarkdownNode
{
@@ -0,0 +1,10 @@
namespace DiscordChatExporter.Core.Markdown
{
internal enum MentionKind
{
Meta,
User,
Channel,
Role
}
}
@@ -0,0 +1,17 @@
namespace DiscordChatExporter.Core.Markdown
{
internal class MentionNode : MarkdownNode
{
public string Id { get; }
public MentionKind Kind { get; }
public MentionNode(string id, MentionKind kind)
{
Id = id;
Kind = kind;
}
public override string ToString() => $"<{Kind} mention> {Id}";
}
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown.Ast
namespace DiscordChatExporter.Core.Markdown
{
internal class MultiLineCodeBlockNode : MarkdownNode
{
@@ -1,6 +1,6 @@
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown.Matching
namespace DiscordChatExporter.Core.Markdown.Parsing
{
internal class AggregateMatcher<T> : IMatcher<T>
{
@@ -1,7 +1,7 @@
using System;
using System.Collections.Generic;
namespace DiscordChatExporter.Core.Markdown.Matching
namespace DiscordChatExporter.Core.Markdown.Parsing
{
internal interface IMatcher<T>
{
@@ -1,10 +1,11 @@
using System.Collections.Generic;
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Text.RegularExpressions;
using DiscordChatExporter.Core.Markdown.Ast;
using DiscordChatExporter.Core.Markdown.Matching;
using DiscordChatExporter.Core.Utils;
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Parsing
{
// The following parsing logic is meant to replicate Discord's markdown grammar as close as possible
internal static partial class MarkdownParser
@@ -103,7 +104,7 @@ namespace DiscordChatExporter.Core.Markdown
// There can be either one or two backticks, but equal number on both sides
private static readonly IMatcher<MarkdownNode> InlineCodeBlockNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(`{1,2})([^`]+)\\1", DefaultRegexOptions | RegexOptions.Singleline),
m => new InlineCodeBlockNode(m.Groups[2].Value.Trim('\r', '\n'))
(_, m) => new InlineCodeBlockNode(m.Groups[2].Value.Trim('\r', '\n'))
);
// Capture language identifier and then any character until the earliest triple backtick
@@ -111,7 +112,7 @@ namespace DiscordChatExporter.Core.Markdown
// Blank lines at the beginning and end of content are trimmed
private static readonly IMatcher<MarkdownNode> MultiLineCodeBlockNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("```(?:(\\w*)\\n)?(.+?)```", DefaultRegexOptions | RegexOptions.Singleline),
m => new MultiLineCodeBlockNode(m.Groups[1].Value, m.Groups[2].Value.Trim('\r', '\n'))
(_, m) => new MultiLineCodeBlockNode(m.Groups[1].Value, m.Groups[2].Value.Trim('\r', '\n'))
);
/* Mentions */
@@ -119,31 +120,31 @@ namespace DiscordChatExporter.Core.Markdown
// Capture @everyone
private static readonly IMatcher<MarkdownNode> EveryoneMentionNodeMatcher = new StringMatcher<MarkdownNode>(
"@everyone",
_ => new MentionNode("everyone", MentionType.Meta)
_ => new MentionNode("everyone", MentionKind.Meta)
);
// Capture @here
private static readonly IMatcher<MarkdownNode> HereMentionNodeMatcher = new StringMatcher<MarkdownNode>(
"@here",
_ => new MentionNode("here", MentionType.Meta)
_ => new MentionNode("here", MentionKind.Meta)
);
// Capture <@123456> or <@!123456>
private static readonly IMatcher<MarkdownNode> UserMentionNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<@!?(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Groups[1].Value, MentionType.User)
(_, m) => new MentionNode(m.Groups[1].Value, MentionKind.User)
);
// Capture <#123456>
private static readonly IMatcher<MarkdownNode> ChannelMentionNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<#(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Groups[1].Value, MentionType.Channel)
new Regex("<#!?(\\d+)>", DefaultRegexOptions),
(_, m) => new MentionNode(m.Groups[1].Value, MentionKind.Channel)
);
// Capture <@&123456>
private static readonly IMatcher<MarkdownNode> RoleMentionNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<@&(\\d+)>", DefaultRegexOptions),
m => new MentionNode(m.Groups[1].Value, MentionType.Role)
(_, m) => new MentionNode(m.Groups[1].Value, MentionKind.Role)
);
/* Emojis */
@@ -154,15 +155,26 @@ namespace DiscordChatExporter.Core.Markdown
// ... or digit followed by enclosing mark
// (this does not match all emojis in Discord but it's reasonably accurate enough)
private static readonly IMatcher<MarkdownNode> StandardEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("((?:[\\uD83C][\\uDDE6-\\uDDFF]){2}|[\\u2600-\\u26FF]|\\p{Cs}{2}|\\d\\p{Me})",
DefaultRegexOptions),
m => new EmojiNode(m.Groups[1].Value)
new Regex("((?:[\\uD83C][\\uDDE6-\\uDDFF]){2}|[\\u2600-\\u26FF]|\\p{Cs}{2}|\\d\\p{Me})", DefaultRegexOptions),
(_, m) => new EmojiNode(m.Groups[1].Value)
);
// Capture :thinking: (but only for known emoji codes)
private static readonly IMatcher<MarkdownNode> CodedStandardEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex(":([\\w_]+):", DefaultRegexOptions),
(_, m) =>
{
var name = EmojiIndex.TryGetName(m.Groups[1].Value);
return name is not null
? new EmojiNode(name)
: null;
}
);
// Capture <:lul:123456> or <a:lul:123456>
private static readonly IMatcher<MarkdownNode> CustomEmojiNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<(a)?:(.+?):(\\d+?)>", DefaultRegexOptions),
m => new EmojiNode(m.Groups[3].Value, m.Groups[2].Value, !string.IsNullOrWhiteSpace(m.Groups[1].Value))
(_, m) => new EmojiNode(m.Groups[3].Value, m.Groups[2].Value, !string.IsNullOrWhiteSpace(m.Groups[1].Value))
);
/* Links */
@@ -170,19 +182,19 @@ namespace DiscordChatExporter.Core.Markdown
// Capture [title](link)
private static readonly IMatcher<MarkdownNode> TitledLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\[(.+?)\\]\\((.+?)\\)", DefaultRegexOptions),
m => new LinkNode(m.Groups[2].Value, m.Groups[1].Value)
(_, m) => new LinkNode(m.Groups[2].Value, m.Groups[1].Value)
);
// Capture any non-whitespace character after http:// or https:// until the last punctuation character or whitespace
private static readonly IMatcher<MarkdownNode> AutoLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(https?://\\S*[^\\.,:;\"\'\\s])", DefaultRegexOptions),
m => new LinkNode(m.Groups[1].Value)
(_, m) => new LinkNode(m.Groups[1].Value)
);
// Same as auto link but also surrounded by angular brackets
private static readonly IMatcher<MarkdownNode> HiddenLinkNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<(https?://\\S*[^\\.,:;\"\'\\s])>", DefaultRegexOptions),
m => new LinkNode(m.Groups[1].Value)
(_, m) => new LinkNode(m.Groups[1].Value)
);
/* Text */
@@ -198,21 +210,41 @@ namespace DiscordChatExporter.Core.Markdown
// This escapes it from matching for emoji
private static readonly IMatcher<MarkdownNode> IgnoredEmojiTextNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("(\\u26A7|\\u2640|\\u2642|\\u2695|\\u267E|\\u00A9|\\u00AE|\\u2122)", DefaultRegexOptions),
m => new TextNode(m.Groups[1].Value)
(_, m) => new TextNode(m.Groups[1].Value)
);
// Capture any "symbol/other" character or surrogate pair preceded by a backslash
// This escapes it from matching for emoji
private static readonly IMatcher<MarkdownNode> EscapedSymbolTextNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\\\(\\p{So}|\\p{Cs}{2})", DefaultRegexOptions),
m => new TextNode(m.Groups[1].Value)
(_, m) => new TextNode(m.Groups[1].Value)
);
// Capture any non-whitespace, non latin alphanumeric character preceded by a backslash
// This escapes it from matching for formatting or other tokens
private static readonly IMatcher<MarkdownNode> EscapedCharacterTextNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("\\\\([^a-zA-Z0-9\\s])", DefaultRegexOptions),
m => new TextNode(m.Groups[1].Value)
(_, m) => new TextNode(m.Groups[1].Value)
);
/* Misc */
// Capture <t:12345678> or <t:12345678:R>
private static readonly IMatcher<MarkdownNode> UnixTimestampNodeMatcher = new RegexMatcher<MarkdownNode>(
new Regex("<t:(\\d+)(?::\\w)?>", DefaultRegexOptions),
(_, m) =>
{
// We don't care about the 'R' parameter because we're not going to
// show relative timestamps in an export anyway.
if (!long.TryParse(m.Groups[1].Value, NumberStyles.Integer, CultureInfo.InvariantCulture,
out var offset))
{
return null;
}
return new UnixTimestampNode(DateTimeOffset.UnixEpoch + TimeSpan.FromSeconds(offset));
}
);
// Combine all matchers into one
@@ -255,7 +287,11 @@ namespace DiscordChatExporter.Core.Markdown
// Emoji
StandardEmojiNodeMatcher,
CustomEmojiNodeMatcher
CustomEmojiNodeMatcher,
CodedStandardEmojiNodeMatcher,
// Misc
UnixTimestampNodeMatcher
);
// Minimal set of matchers for non-multimedia formats (e.g. plain text)
@@ -268,7 +304,10 @@ namespace DiscordChatExporter.Core.Markdown
RoleMentionNodeMatcher,
// Emoji
CustomEmojiNodeMatcher
CustomEmojiNodeMatcher,
// Misc
UnixTimestampNodeMatcher
);
private static IReadOnlyList<MarkdownNode> Parse(StringPart stringPart, IMatcher<MarkdownNode> matcher) =>
@@ -280,12 +319,16 @@ namespace DiscordChatExporter.Core.Markdown
internal static partial class MarkdownParser
{
private static IReadOnlyList<MarkdownNode> Parse(StringPart stringPart) => Parse(stringPart, AggregateNodeMatcher);
private static IReadOnlyList<MarkdownNode> Parse(StringPart stringPart) =>
Parse(stringPart, AggregateNodeMatcher);
private static IReadOnlyList<MarkdownNode> ParseMinimal(StringPart stringPart) => Parse(stringPart, MinimalAggregateNodeMatcher);
private static IReadOnlyList<MarkdownNode> ParseMinimal(StringPart stringPart) =>
Parse(stringPart, MinimalAggregateNodeMatcher);
public static IReadOnlyList<MarkdownNode> Parse(string input) => Parse(new StringPart(input));
public static IReadOnlyList<MarkdownNode> Parse(string input) =>
Parse(new StringPart(input));
public static IReadOnlyList<MarkdownNode> ParseMinimal(string input) => ParseMinimal(new StringPart(input));
public static IReadOnlyList<MarkdownNode> ParseMinimal(string input) =>
ParseMinimal(new StringPart(input));
}
}
@@ -1,12 +1,12 @@
using System;
using System.Collections.Generic;
using DiscordChatExporter.Core.Markdown.Ast;
namespace DiscordChatExporter.Core.Markdown
namespace DiscordChatExporter.Core.Markdown.Parsing
{
internal abstract class MarkdownVisitor
{
protected virtual MarkdownNode VisitText(TextNode text) => text;
protected virtual MarkdownNode VisitText(TextNode text) =>
text;
protected virtual MarkdownNode VisitFormatted(FormattedNode formatted)
{
@@ -14,15 +14,23 @@ namespace DiscordChatExporter.Core.Markdown
return formatted;
}
protected virtual MarkdownNode VisitInlineCodeBlock(InlineCodeBlockNode inlineCodeBlock) => inlineCodeBlock;
protected virtual MarkdownNode VisitInlineCodeBlock(InlineCodeBlockNode inlineCodeBlock) =>
inlineCodeBlock;
protected virtual MarkdownNode VisitMultiLineCodeBlock(MultiLineCodeBlockNode multiLineCodeBlock) => multiLineCodeBlock;
protected virtual MarkdownNode VisitMultiLineCodeBlock(MultiLineCodeBlockNode multiLineCodeBlock) =>
multiLineCodeBlock;
protected virtual MarkdownNode VisitLink(LinkNode link) => link;
protected virtual MarkdownNode VisitLink(LinkNode link) =>
link;
protected virtual MarkdownNode VisitEmoji(EmojiNode emoji) => emoji;
protected virtual MarkdownNode VisitEmoji(EmojiNode emoji) =>
emoji;
protected virtual MarkdownNode VisitMention(MentionNode mention) => mention;
protected virtual MarkdownNode VisitMention(MentionNode mention) =>
mention;
protected virtual MarkdownNode VisitUnixTimestamp(UnixTimestampNode timestamp) =>
timestamp;
public MarkdownNode Visit(MarkdownNode node) => node switch
{
@@ -33,6 +41,7 @@ namespace DiscordChatExporter.Core.Markdown
LinkNode link => VisitLink(link),
EmojiNode emoji => VisitEmoji(emoji),
MentionNode mention => VisitMention(mention),
UnixTimestampNode timestamp => VisitUnixTimestamp(timestamp),
_ => throw new ArgumentOutOfRangeException(nameof(node))
};
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown.Matching
namespace DiscordChatExporter.Core.Markdown.Parsing
{
internal class ParsedMatch<T>
{
@@ -1,24 +1,19 @@
using System;
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Markdown.Matching
namespace DiscordChatExporter.Core.Markdown.Parsing
{
internal class RegexMatcher<T> : IMatcher<T>
{
private readonly Regex _regex;
private readonly Func<StringPart, Match, T> _transform;
private readonly Func<StringPart, Match, T?> _transform;
public RegexMatcher(Regex regex, Func<StringPart, Match, T> transform)
public RegexMatcher(Regex regex, Func<StringPart, Match, T?> transform)
{
_regex = regex;
_transform = transform;
}
public RegexMatcher(Regex regex, Func<Match, T> transform)
: this(regex, (p, m) => transform(m))
{
}
public ParsedMatch<T>? TryMatch(StringPart stringPart)
{
var match = _regex.Match(stringPart.Target, stringPart.StartIndex, stringPart.Length);
@@ -30,11 +25,15 @@ namespace DiscordChatExporter.Core.Markdown.Matching
// Which is super weird because regex.Match(string, int) takes the whole input in context.
// So in order to properly account for ^/$ regex tokens, we need to make sure that
// the expression also matches on the bigger part of the input.
if (!_regex.IsMatch(stringPart.Target.Substring(0, stringPart.EndIndex), stringPart.StartIndex))
if (!_regex.IsMatch(stringPart.Target[..stringPart.EndIndex], stringPart.StartIndex))
return null;
var stringPartMatch = stringPart.Slice(match.Index, match.Length);
return new ParsedMatch<T>(stringPartMatch, _transform(stringPartMatch, match));
var value = _transform(stringPartMatch, match);
return value is not null
? new ParsedMatch<T>(stringPartMatch, value)
: null;
}
}
}
@@ -1,14 +1,14 @@
using System;
namespace DiscordChatExporter.Core.Markdown.Matching
namespace DiscordChatExporter.Core.Markdown.Parsing
{
internal class StringMatcher<T> : IMatcher<T>
{
private readonly string _needle;
private readonly StringComparison _comparison;
private readonly Func<StringPart, T> _transform;
private readonly Func<StringPart, T?> _transform;
public StringMatcher(string needle, StringComparison comparison, Func<StringPart, T> transform)
public StringMatcher(string needle, StringComparison comparison, Func<StringPart, T?> transform)
{
_needle = needle;
_comparison = comparison;
@@ -23,14 +23,15 @@ namespace DiscordChatExporter.Core.Markdown.Matching
public ParsedMatch<T>? TryMatch(StringPart stringPart)
{
var index = stringPart.Target.IndexOf(_needle, stringPart.StartIndex, stringPart.Length, _comparison);
if (index < 0)
return null;
if (index >= 0)
{
var stringPartMatch = stringPart.Slice(index, _needle.Length);
return new ParsedMatch<T>(stringPartMatch, _transform(stringPartMatch));
}
var stringPartMatch = stringPart.Slice(index, _needle.Length);
var value = _transform(stringPartMatch);
return null;
return value is not null
? new ParsedMatch<T>(stringPartMatch, value)
: null;
}
}
}
@@ -1,6 +1,6 @@
using System.Text.RegularExpressions;
namespace DiscordChatExporter.Core.Markdown.Matching
namespace DiscordChatExporter.Core.Markdown.Parsing
{
internal readonly struct StringPart
{
@@ -10,14 +10,13 @@ namespace DiscordChatExporter.Core.Markdown.Matching
public int Length { get; }
public int EndIndex { get; }
public int EndIndex => StartIndex + Length;
public StringPart(string target, int startIndex, int length)
{
Target = target;
StartIndex = startIndex;
Length = length;
EndIndex = startIndex + length;
}
public StringPart(string target)
@@ -0,0 +1,12 @@
namespace DiscordChatExporter.Core.Markdown
{
internal enum TextFormatting
{
Bold,
Italic,
Underline,
Strikethrough,
Spoiler,
Quote
}
}
@@ -1,4 +1,4 @@
namespace DiscordChatExporter.Core.Markdown.Ast
namespace DiscordChatExporter.Core.Markdown
{
internal class TextNode : MarkdownNode
{
@@ -0,0 +1,13 @@
using System;
namespace DiscordChatExporter.Core.Markdown
{
internal class UnixTimestampNode : MarkdownNode
{
public DateTimeOffset Value { get; }
public UnixTimestampNode(DateTimeOffset value) => Value = value;
public override string ToString() => Value.ToString();
}
}
File diff suppressed because it is too large Load Diff
@@ -9,6 +9,6 @@ namespace DiscordChatExporter.Core.Utils.Extensions
public static T? NullIf<T>(this T value, Func<T, bool> predicate) where T : struct =>
!predicate(value)
? value
: (T?) null;
: null;
}
}
@@ -1,14 +1,30 @@
using System.Text;
using System.Collections.Generic;
using System.Text;
namespace DiscordChatExporter.Core.Utils.Extensions
{
public static class StringExtensions
{
public static string? NullIfWhiteSpace(this string str) =>
!string.IsNullOrWhiteSpace(str)
? str
: null;
public static string Truncate(this string str, int charCount) =>
str.Length > charCount
? str.Substring(0, charCount)
? str[..charCount]
: str;
public static IEnumerable<Rune> GetRunes(this string str)
{
var lastIndex = 0;
while (lastIndex < str.Length && Rune.TryGetRuneAt(str, lastIndex, out var rune))
{
yield return rune;
lastIndex += rune.Utf16SequenceLength;
}
}
public static StringBuilder AppendIfNotEmpty(this StringBuilder builder, char value) =>
builder.Length > 0
? builder.Append(value)
@@ -0,0 +1,24 @@
using System;
using Superpower;
using Superpower.Parsers;
namespace DiscordChatExporter.Core.Utils.Extensions
{
public static class SuperpowerExtensions
{
public static TextParser<string> Text(this TextParser<char[]> parser) =>
parser.Select(chars => new string(chars));
public static TextParser<T> Token<T>(this TextParser<T> parser) =>
parser.Between(Character.WhiteSpace.IgnoreMany(), Character.WhiteSpace.IgnoreMany());
// From: https://twitter.com/nblumhardt/status/1389349059786264578
public static TextParser<T> Log<T>(this TextParser<T> parser, string description) => i =>
{
Console.WriteLine($"Trying {description} ->");
var r = parser(i);
Console.WriteLine($"Result was {r}");
return r;
};
}
}
+20 -14
View File
@@ -5,6 +5,7 @@ using System.Net;
using System.Net.Http;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using DiscordChatExporter.Core.Utils.Extensions;
using Polly;
namespace DiscordChatExporter.Core.Utils
@@ -20,14 +21,15 @@ namespace DiscordChatExporter.Core.Utils
.OrResult<HttpResponseMessage>(m => m.StatusCode == HttpStatusCode.TooManyRequests)
.OrResult(m => m.StatusCode == HttpStatusCode.RequestTimeout)
.OrResult(m => m.StatusCode >= HttpStatusCode.InternalServerError)
.WaitAndRetryAsync(8,
.WaitAndRetryAsync(
8,
(i, result, _) =>
{
// If rate-limited, use retry-after as a guide
if (result.Result?.StatusCode == HttpStatusCode.TooManyRequests)
{
// Only start respecting retry-after after a few attempts.
// The reason is that Discord often sends unreasonable (20+ minutes) retry-after
// Only start respecting retry-after after a few attempts, because
// Discord often sends unreasonable (20+ minutes) retry-after
// on the very first request.
if (i > 3)
{
@@ -39,23 +41,27 @@ namespace DiscordChatExporter.Core.Utils
return TimeSpan.FromSeconds(Math.Pow(2, i) + 1);
},
(_, _, _, _) => Task.CompletedTask);
(_, _, _, _) => Task.CompletedTask
);
private static HttpStatusCode? TryGetStatusCodeFromException(HttpRequestException ex)
{
private static HttpStatusCode? TryGetStatusCodeFromException(HttpRequestException ex) =>
// This is extremely frail, but there's no other way
var statusCodeRaw = Regex.Match(ex.Message, @": (\d+) \(").Groups[1].Value;
return !string.IsNullOrWhiteSpace(statusCodeRaw)
? (HttpStatusCode) int.Parse(statusCodeRaw, CultureInfo.InvariantCulture)
: (HttpStatusCode?) null;
}
Regex
.Match(ex.Message, @": (\d+) \(")
.Groups[1]
.Value
.NullIfWhiteSpace()?
.Pipe(s => (HttpStatusCode) int.Parse(s, CultureInfo.InvariantCulture));
public static IAsyncPolicy ExceptionPolicy { get; } =
Policy
.Handle<IOException>() // dangerous
.Or<HttpRequestException>(ex => TryGetStatusCodeFromException(ex) == HttpStatusCode.TooManyRequests)
.Or<HttpRequestException>(ex => TryGetStatusCodeFromException(ex) == HttpStatusCode.RequestTimeout)
.Or<HttpRequestException>(ex => TryGetStatusCodeFromException(ex) >= HttpStatusCode.InternalServerError)
.Or<HttpRequestException>(ex =>
TryGetStatusCodeFromException(ex) is
HttpStatusCode.TooManyRequests or
HttpStatusCode.RequestTimeout or
HttpStatusCode.InternalServerError
)
.WaitAndRetryAsync(4, i => TimeSpan.FromSeconds(Math.Pow(2, i) + 1));
}
}
@@ -0,0 +1,9 @@
// ReSharper disable CheckNamespace
// TODO: remove after moving to .NET 5
namespace System.Runtime.CompilerServices
{
internal static class IsExternalInit
{
}
}
@@ -11,9 +11,13 @@
<Resource Include="../favicon.ico" />
</ItemGroup>
<ItemGroup Condition="'$(Configuration)' == 'Release'">
<PackageReference Include="DotnetRuntimeBootstrapper" Version="1.1.2" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Gress" Version="1.2.0" />
<PackageReference Include="MaterialDesignColors" Version="2.0.0" />
<PackageReference Include="MaterialDesignColors" Version="2.0.1" />
<PackageReference Include="MaterialDesignThemes" Version="4.0.0" />
<PackageReference Include="Microsoft.Xaml.Behaviors.Wpf" Version="1.1.31" />
<PackageReference Include="Ookii.Dialogs.Wpf" Version="3.1.0" />
@@ -21,7 +25,7 @@
<PackageReference Include="Stylet" Version="1.3.6" />
<PackageReference Include="Tyrrrz.Extensions" Version="1.6.5" />
<PackageReference Include="Tyrrrz.Settings" Version="1.3.4" />
<PackageReference Include="PropertyChanged.Fody" Version="3.3.3" PrivateAssets="all" />
<PackageReference Include="PropertyChanged.Fody" Version="3.4.0" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
@@ -25,6 +25,8 @@ namespace DiscordChatExporter.Gui.Services
public string? LastPartitionLimitValue { get; set; }
public string? LastMessageFilterValue { get; set; }
public bool LastShouldDownloadMedia { get; set; }
public SettingsService()
@@ -4,6 +4,7 @@ using System.Linq;
using DiscordChatExporter.Core.Discord;
using DiscordChatExporter.Core.Discord.Data;
using DiscordChatExporter.Core.Exporting;
using DiscordChatExporter.Core.Exporting.Filtering;
using DiscordChatExporter.Core.Exporting.Partitioning;
using DiscordChatExporter.Core.Utils.Extensions;
using DiscordChatExporter.Gui.Services;
@@ -51,7 +52,13 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
public PartitionLimit PartitionLimit => !string.IsNullOrWhiteSpace(PartitionLimitValue)
? PartitionLimit.Parse(PartitionLimitValue)
: NullPartitionLimit.Instance;
: PartitionLimit.Null;
public string? MessageFilterValue { get; set; }
public MessageFilter MessageFilter => !string.IsNullOrWhiteSpace(MessageFilterValue)
? MessageFilter.Parse(MessageFilterValue)
: MessageFilter.Null;
public bool ShouldDownloadMedia { get; set; }
@@ -61,6 +68,7 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
After != default ||
Before != default ||
!string.IsNullOrWhiteSpace(PartitionLimitValue) ||
!string.IsNullOrWhiteSpace(MessageFilterValue) ||
ShouldDownloadMedia != default;
public ExportSetupViewModel(DialogManager dialogManager, SettingsService settingsService)
@@ -71,6 +79,7 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
// Persist preferences
SelectedFormat = _settingsService.LastExportFormat;
PartitionLimitValue = _settingsService.LastPartitionLimitValue;
MessageFilterValue = _settingsService.LastMessageFilterValue;
ShouldDownloadMedia = _settingsService.LastShouldDownloadMedia;
}
@@ -79,6 +88,7 @@ namespace DiscordChatExporter.Gui.ViewModels.Dialogs
// Persist preferences
_settingsService.LastExportFormat = SelectedFormat;
_settingsService.LastPartitionLimitValue = PartitionLimitValue;
_settingsService.LastMessageFilterValue = MessageFilterValue;
_settingsService.LastShouldDownloadMedia = ShouldDownloadMedia;
// If single channel - prompt file path
@@ -0,0 +1,27 @@
using DiscordChatExporter.Gui.ViewModels.Framework;
namespace DiscordChatExporter.Gui.ViewModels.Dialogs
{
public class MessageBoxViewModel : DialogScreen
{
public string? Title { get; set; }
public string? Message { get; set; }
}
public static class MessageBoxViewModelExtensions
{
public static MessageBoxViewModel CreateMessageBoxViewModel(
this IViewModelFactory factory,
string title,
string message)
{
var viewModel = factory.CreateMessageBoxViewModel();
viewModel.Title = title;
viewModel.Message = message;
return viewModel;
}
}
}
@@ -7,6 +7,8 @@ namespace DiscordChatExporter.Gui.ViewModels.Framework
{
ExportSetupViewModel CreateExportSetupViewModel();
MessageBoxViewModel CreateMessageBoxViewModel();
SettingsViewModel CreateSettingsViewModel();
}
}
@@ -168,7 +168,8 @@ namespace DiscordChatExporter.Gui.ViewModels
var guildChannelMap = new Dictionary<Guild, IReadOnlyList<Channel>>();
await foreach (var guild in discord.GetUserGuildsAsync())
{
guildChannelMap[guild] = await discord.GetGuildChannelsAsync(guild.Id);
var channels = await discord.GetGuildChannelsAsync(guild.Id);
guildChannelMap[guild] = channels.Where(c => c.IsTextChannel).ToArray();
}
GuildChannelMap = guildChannelMap;
@@ -178,6 +179,15 @@ namespace DiscordChatExporter.Gui.ViewModels
{
Notifications.Enqueue(ex.Message.TrimEnd('.'));
}
catch (Exception ex)
{
var dialog = _viewModelFactory.CreateMessageBoxViewModel(
"Error pulling guilds and channels",
ex.ToString()
);
await _dialogManager.ShowDialogAsync(dialog);
}
}
public bool CanExportChannels =>
@@ -185,55 +195,68 @@ namespace DiscordChatExporter.Gui.ViewModels
public async void ExportChannels()
{
var token = _settingsService.LastToken;
if (token is null || SelectedGuild is null || SelectedChannels is null || !SelectedChannels.Any())
return;
var dialog = _viewModelFactory.CreateExportSetupViewModel(SelectedGuild, SelectedChannels);
if (await _dialogManager.ShowDialogAsync(dialog) != true)
return;
var exporter = new ChannelExporter(token);
var operations = ProgressManager.CreateOperations(dialog.Channels!.Count);
var successfulExportCount = 0;
await dialog.Channels.Zip(operations).ParallelForEachAsync(async tuple =>
try
{
var (channel, operation) = tuple;
var token = _settingsService.LastToken;
if (token is null || SelectedGuild is null || SelectedChannels is null || !SelectedChannels.Any())
return;
try
var dialog = _viewModelFactory.CreateExportSetupViewModel(SelectedGuild, SelectedChannels);
if (await _dialogManager.ShowDialogAsync(dialog) != true)
return;
var exporter = new ChannelExporter(token);
var operations = ProgressManager.CreateOperations(dialog.Channels!.Count);
var successfulExportCount = 0;
await dialog.Channels.Zip(operations).ParallelForEachAsync(async tuple =>
{
var request = new ExportRequest(
dialog.Guild!,
channel!,
dialog.OutputPath!,
dialog.SelectedFormat,
dialog.After?.Pipe(Snowflake.FromDate),
dialog.Before?.Pipe(Snowflake.FromDate),
dialog.PartitionLimit,
dialog.ShouldDownloadMedia,
_settingsService.ShouldReuseMedia,
_settingsService.DateFormat
);
var (channel, operation) = tuple;
await exporter.ExportChannelAsync(request, operation);
try
{
var request = new ExportRequest(
dialog.Guild!,
channel!,
dialog.OutputPath!,
dialog.SelectedFormat,
dialog.After?.Pipe(Snowflake.FromDate),
dialog.Before?.Pipe(Snowflake.FromDate),
dialog.PartitionLimit,
dialog.MessageFilter,
dialog.ShouldDownloadMedia,
_settingsService.ShouldReuseMedia,
_settingsService.DateFormat
);
Interlocked.Increment(ref successfulExportCount);
}
catch (DiscordChatExporterException ex) when (!ex.IsCritical)
{
Notifications.Enqueue(ex.Message.TrimEnd('.'));
}
finally
{
operation.Dispose();
}
}, _settingsService.ParallelLimit.ClampMin(1));
await exporter.ExportChannelAsync(request, operation);
// Notify of overall completion
if (successfulExportCount > 0)
Notifications.Enqueue($"Successfully exported {successfulExportCount} channel(s)");
Interlocked.Increment(ref successfulExportCount);
}
catch (DiscordChatExporterException ex) when (!ex.IsCritical)
{
Notifications.Enqueue(ex.Message.TrimEnd('.'));
}
finally
{
operation.Dispose();
}
}, _settingsService.ParallelLimit.ClampMin(1));
// Notify of overall completion
if (successfulExportCount > 0)
Notifications.Enqueue($"Successfully exported {successfulExportCount} channel(s)");
}
catch (Exception ex)
{
var dialog = _viewModelFactory.CreateMessageBoxViewModel(
"Error exporting channel(s)",
ex.ToString()
);
await _dialogManager.ShowDialogAsync(dialog);
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More