Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds better abilities to check, what exactly was rate limited #2901

Open
wants to merge 162 commits into
base: next
Choose a base branch
from

Conversation

TheCataliasTNT2k
Copy link

Add "OwnedRoute" to "RatelimitInfo" which allows to check the cause of the rate limit and which identifiers were affected (e.g. which channel).

@TheCataliasTNT2k TheCataliasTNT2k changed the base branch from current to next June 12, 2024 22:47
@github-actions github-actions bot added the http Related to the `http` module. label Jun 12, 2024
@TheCataliasTNT2k
Copy link
Author

The failing check seems broken, at least I do not get any error when looking at the details...

@jamesbt365
Copy link
Member

The failing check seems broken, at least I do not get any error when looking at the details...

cargo +nightly fmt --all

src/http/routing.rs Outdated Show resolved Hide resolved
src/http/routing.rs Outdated Show resolved Hide resolved
Comment on lines +36 to +39
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
enum RouteKind {
$($name,)+
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this moved?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To have two similar blocks next to another (Route and OwnedRoute).
I could move it back, but in my opinion it is more readable like this.

/// please match the variants you are interested in.
#[must_use]
#[allow(unused_variables)] // prevent compiler from complaining about unused variables
pub fn get_common_identifiers(&self) -> RatelimitCause {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub fn get_common_identifiers(&self) -> RatelimitCause {
pub fn cause(&self) -> RatelimitCause {

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not really return the "cause" of the rate limit but the parts of the path, which was rate limited.
The "cause" would be the deletion of a message or something like that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the type should not be caused RatelimitCause, it should be called RatelimitPathParts or something else.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is better, yes.
Can the method name stay then?

@@ -27,23 +27,24 @@ enum RatelimitingKind {
macro_rules! routes {
($lt:lifetime, {
$(
$name:ident $({ $($field_name:ident: $field_type:ty),* })?,
$name:ident $({ $($field_name_route:ident: $field_type:ty | $field_type_owned:ty),* })?,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of making the macro syntax messier, can you make this use traits instead? I'm pretty sure ToOwned should handle this, but if it doesn't you can just write a simple private trait.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How? I somehow need to know the type, which the OwnedRoute should use. I do not know, how a trait would be able to do that.
The problem is the "&str" type. Can you please provide an example for how to do it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The owned type is simply <$field_type as ToOwned>::Owned, and if that doesn't return the correct types you can duplicate the definition of ToOwned and implement it for the types needed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do that.

Copy link
Author

@TheCataliasTNT2k TheCataliasTNT2k Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The field_type_owned is actually needed, I did not find a way to get compiling code without it.

Reason:
The macro would expand into something like this (irrelevant code removed):

#[derive(Clone, Debug)]
pub enum OwnedRoute {
    ChannelMessageReaction { channel_id: <ChannelId as ToOwned>::Owned, message_id: <MessageId as ToOwned>::Owned, user_id: <UserId as ToOwned>::Owned, reaction: <&'a str as ToOwned>::Owned},
}

<&'a str as ToOwned>::Owned is not valid, because 'a does not exist. I could add the lifetime to the enum, but then we would need to carry this useless lifetime though the whole code. This can not be fixed by replacing the <&'a str as ToOwned>::Owned with a custom macro, I already tried:

macro_rules! get_owned_type {
    (&'a str) => {String};
    ($type:ty) => {<$type as ToOwned>::Owned};
}

#[derive(Clone, Debug)]
pub enum OwnedRoute {
    ChannelMessageReaction { channel_id: get_owned_type!(ChannelId), message_id: get_owned_type!(MessageId), user_id: get_owned_type!(UserId), reaction: get_owned_type!(&'a str)},
}

will not compile for the same reason.

The compiler is not smart enough to ignore the unused lifetime at the moment.

}

#[must_use]
pub fn get_owned_route(&self) -> OwnedRoute {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rust convention is to name getters without the prefix, but since this is a &self -> Owned I think the convention would be to_owned_route... however that's just ToOwned, so implement ToOwned instead.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not use toOwned, because this is not really a clone and should not be treated as one, which is expected by the trait however.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With that logic ToOwned should not be a thing, but this is exactly what it's for. This is internally just converting str -> String which is exactly the point of ToOwned.

Copy link
Author

@TheCataliasTNT2k TheCataliasTNT2k Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can not be done using ToOwned, because this would require the "copied" object to carry the lifetime with it.

I already tried that sometime, and it makes the whole code a mess.

I renamed the method though (not pushed yet because of the other change requests).

GnomedDev and others added 17 commits November 15, 2024 19:08
…erenity-rs#2646)

This avoids having to allocate to store fixed length (replaced with normal
array) or fixed capacity (replaced with `ArrayVec`) collections as vectors for
the purposes of putting them through the `Request` plumbing.

Slight behavioral change - before, setting `params` to `Some(vec![])`
would still append a question mark to the end of the url. Now, we check
if the params array `is_empty` instead of `is_some`, so the question
mark won't be appended if the params list is empty.

Co-authored-by: Michael Krasnitski <[email protected]>
These are unnecessary. Accepting `impl Into<Arc<T>>` allows passing either `T` or `Arc<T>`.
This trades a heap allocation for messages sent along with thread
creation for `Message`'s inline size dropping from 1176 bytes to 760
bytes,
…l models (serenity-rs#2656)

This shrinks type sizes by a lot; however, it makes the user experience slightly
different:

- `FixedString` must be converted to String with `.into()` or `.into_string()`
  before it can be pushed to, but dereferences to `&str` as is.
- `FixedArray` must be converted to `Vec` with `.into()` or `.into_vec()`
  before it can be pushed to, but dereferences to `&[T]` as is.

The crate of these types is currently a Git dependency, but this is fine for
the `next` branch. It needs some basic testing, which Serenity is perfect for,
before a release will be made to crates.io.
…enity-rs#2668)

This commit:

- switches from `u64` to `i64` in `CreateCommandOption::min_int_value` and
`CreateCommandOption::max_int_value` to accommodate negative integers in
Discord's integer range (between -2^53 and 2^53). Values outside this
range will cause Discord's API to return an error.
- switches from `i32` to `i64` in `CreateCommandOption::add_int_choice` and
`CreateCommandOption::add_int_choice_localized` to accommodate Discord's
complete integer range (between -2^53 and 2^53). Values outside this
range will cause Discord's API to return an error.
This cache was just duplicating information already present in `Guild::members`
and therefore should be removed.

This saves around 700 MBs for my bot (pre-`FixedString`).

This has to refactor `utils::content_safe` to always take a `Guild` instead
of`Cache`, but in practice it was mostly pulling from the guild cache anyway
and this means it is more likely to respect nicknames and other information,
while losing the ability to clean mentions from DMs, which do not matter.
`Embed::fields` previously had to stay as a `Vec` due to `CreateEmbed` wrapping
around it, but by implementing `Serialize` manually we can overwrite the
`Embed::fields` with a normal `Vec`, for a small performance hit on
serialization while saving some space for all stored `Embed`s.
Simply missed these when finding and replacing.
This uses the `bool_to_bitflags` macro to remove boolean (and optional boolean)
fields from structs and pack them into a bitflags invocation, so a struct with
many bools will only use one or two bytes, instead of a byte per bool as is.

This requires using getters and setters for the boolean fields, which changes
user experience and is hard to document, which is a significant downside, but
is such a nice change and will just become more and more efficient as time goes
on.
…rs#2681)

This swaps fields that store `Option<Int>` for `Option<NonMaxInt>` where the
maximum value would be ludicrous. Since `nonmax` uses `NonZero` internally,
this gives us niche optimisations, so model sizes can drop some more.

I have had to include a workaround for [serenity-rs#17] in `optional_string` by making my
own `TryFrom<u64>`, so that should be removable once that issue is fixed.

[serenity-rs#17]: LPGhatguy/nonmax#17
A couple of clippy bugs have been fixed and I have shrunk model
sizes enough to make `clippy::large_enum_variant` go away.
A discord bot library should not be using the tools reserved for low
level OS interaction/data structure libraries.
GnomedDev and others added 10 commits November 15, 2024 19:21
…s#3000)

More type safe, more performant, less reliant on serde_json, what's not
to love!
This is needed for use cases like returning a borrow into the user data
from a function which takes a reference to Context.
This deletes methods which simply pass through to their `id` fields, as
they are:
- Misleading, as a user may do an HTTP call or cache lookup to get a
model just to use it's ID
- Lead to a bunch of maintenance issues when having 2..=4 copies of
every method.
- Hurt compile times, as rustc is having to check signatures, calls,
construct async state machines, and maybe even inline to generate code
multiple times.

For methods which simply only used their `id` field but did not have a
version on their respective ID, they were moved to their ID.

This doesn't have a corresponding deprecation PR to current, as current
still has permission checks and other "helpers" which rely on the model.
This exposes the `bytes` dependency but in turn allows for people to
pass responses directly from reqwest to discord without cloning the
data, as well as allowing reuploading cached data without cloning or
leaking to get a static reference.

The bytes dependency also has to be made non-optional, but this is fine
as it was already due to `aformat` depending on it via `bytestring`.
This separates the the builders for creating versus editing a command,
since it's not possible to change the `type` of a command (in serenity
this is the `kind` field). Also, the `name` field is not required when
editing a command.
Improved wording and removed some sentences that are no longer relevant.
These docs are mostly the ones that were touched by serenity-rs#2988.
Discord documents this as always present, even with Ping interactions.
GnomedDev and others added 9 commits November 16, 2024 12:49
This follows up on serenity-rs#3037, and also removes the now unnecessary Option
around the GuildChannel param.
Turns out this was always being passed 204, except one case where it
should have been deserialising the `Member` return type anyway. This
fixes both issues.
This allows it to imported via `poise::serenity_prelude` which is often
`use`'d as `serenity`.
Copy link
Member

@GnomedDev GnomedDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot review this PR without it being rebased correctly on-top of next.

@TheCataliasTNT2k TheCataliasTNT2k force-pushed the better_ratelimits branch 2 times, most recently from cf3701f to ec3a62e Compare November 25, 2024 00:17
@TheCataliasTNT2k
Copy link
Author

I cannot review this PR without it being rebased correctly on-top of next.

Was doing that right as you wrote that, lol xD

It is just a copy of Route without any lifetimes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
http Related to the `http` module.
Projects
None yet
Development

Successfully merging this pull request may close these issues.