[feature] Guard rails when configuring server limits

[nikclayton]

Hi folks,

You may want to consider putting some guard rails in place when the server's configuration is processed, just to ensure the user hasn't made any potentially costly mistakes.

For example, https://constellation.home.nora.codes/api/v2/instance reports some very large limits, including:

statuses
max_characters	4294967296
max_media_attachments	1024

image_size_limit	4294967296

polls
max_options	1024
max_characters_per_option	1024
min_expiration	300
max_expiration	2629746

I appreciate you can't do anything if the admin puts new limits in to the source code (and I don't know if that's what was done here), but it's probably good practice to sanity check these and other limits just in case someone's made a mistake (e.g., an admin adding an extra 0 to an image or video size limit).

[trysdyn]

There is an upper bound on some of these fields at which point other instances might start dropping activities. I know some source diving in Pleroma ages ago surfaced that it had a rather tight bio length limit at which point it'd just drop any activity from a non-compliant user.

That was long enough ago now that I consider it immaterial for current planning, but it's an anecdote.

A warning of some kind that you're up in the ranges where you might have federation issues might be a consideration? Figuring out what those ranges are is non-trivial though.

[daenney]

I don't really see any of these values as a problem, and we can't determine whether this was the admin's intent or happened by accident.

Some of these will cause federation issues, but we already provide guidance around that in the configuration documentation.

gotosocial/example/config.yaml

Lines 453 to 461 in fcecd0c

	# Int. Maximum allowed image upload size in bytes.
	# Examples: [2097152, 10485760]
	# Default: 10485760 -- aka 10MB
	media-image-max-size: 10485760
	# Int. Maximum allowed video upload size in bytes.
	# Examples: [2097152, 10485760]
	# Default: 41943040 -- aka 40MB
	media-video-max-size: 41943040

We should improve the docs on that one, and make it so you don't have to set it in bytes because that's a bit tedious.

gotosocial/example/config.yaml

Lines 591 to 598 in 504c4f2

	# Int. Maximum amount of characters permitted for a new status,
	# including the content warning (if set).
	#
	# Note that going way higher than the default might break federation.
	#
	# Examples: [140, 500, 5000]
	# Default: 5000
	statuses-max-chars: 5000

gotosocial/example/config.yaml

Lines 612 to 616 in 504c4f2

	# Int. Maximum amount of media files that can be attached to a new status.
	# Note that going way higher than the default might break federation.
	# Examples: [4, 6, 10]
	# Default: 6
	statuses-media-max-files: 6

gotosocial/example/config.yaml

Lines 600 to 604 in 504c4f2

	# Int. Maximum amount of options to permit when creating a new poll.
	# Note that going way higher than the default might break federation.
	# Examples: [4, 6, 10]
	# Default: 6
	statuses-poll-max-options: 6

gotosocial/example/config.yaml

Lines 606 to 610 in 504c4f2

	# Int. Maximum amount of characters to permit per poll option when creating a new poll.
	# Note that going way higher than the default might break federation.
	# Examples: [50, 100, 150]
	# Default: 50
	statuses-poll-option-max-chars: 50

4294967296 is MAX_INT32 and isn't a value you fat-finger. Same with 1024. If people have chosen to explicitly set those values despite what the documentation tells them, then we should let them. Emitting extra warnings about it isn't likely to be helpful.

min_expiration and max_expiration on polls are afaik not configurable by an admin, but I don't see any immediate issue with it being 5 minutes and 30 days respectively.

If there's any evidence of someone having accidentally misconfigured a setting I'd like to hear it, but that seems pretty unlikely given what's shown here.

[NoraCodes]

Hi! I'm the admin and sole user of constellation.home.nora.codes. Let me help you determine whether this was the admin's intent: it was! Don't worry that you'll need to deal with federation issues from these values, as this server runs in allowlist mode, and federates with exactly two other servers.

I'd be curious to learn how these values made it into a client of yours given this extremely limited federation status.

[nikclayton]

Hi @NoraCodes

I'd be curious to learn how these values made it into a client of yours given this extremely limited federation status.

Sure. I'm working on https://pachli.app, and recently changed JSON parsers. The previous parser (Gson) would accept roughly anything you threw at it, if necessary ignoring the type constraints of the object the parsed data was being stored in. This lead to interesting bugs like non-null types containing nulls, trying to store a long in an int, and so on.

This could cause crashes in the app, but those crashes could occur long after the data had been received. It also meant it was difficult to look at the code and reason about its behaviour because Gson could do an end-run around the type system (!)

The new parser (Moshi) is a lot stricter about catching this stuff and reporting it as an error.

Before I rolled out a new version of the app with this change I thought it was prudent to determine how many servers out there had data that could cause problems.

To do that I used the https://fediverse.observer/ API to fetch a list of all servers running Mastodon or Mastodon-like software, and fetched the /api/v1/instance and /api/v2/instance JSON from those servers, then ran those through the parser to check for parse errors.

Doing this showed:

1. There are a number of bugs in existing servers where mandatory data is missing, and a default will need to be provided. Skim through the "//" comments in https://github.com/pachli/pachli-android/blob/main/core/network/src/main/kotlin/app/pachli/core/network/model/InstanceV2.kt for examples.
2. Some servers have set extreme values for some of these options.

E.g., a server that sets 3.1536e22 as the max_toot_chars property, or one that lies about the admin's follower numbers, reporting 97000000048.

Some of those -- e.g., those two examples -- are just people being jerks. Others, e.g., yours where max_characters is exactly 4GiB looked like they could be mistakes. I appreciate that you may have decided that it's OK for your server to support a single post containing approximately one-fifth of Wikipedia, but it's still a bit odd.

Unrestricted / very large content on Mastodon can be an issue for server admins, users, and client developers. mastodon/mastodon#26176 is an example of that.

So when I saw your server configured that way (complete list of weirdness -- https://gist.github.com/nikclayton/ebba4c76261b377094d31fe72f503ec9) I figured it was worth flagging, just in case this was a mistake.

[daenney]

I've pushed #2706 to update the documentation for media sizes to warn about sizes. It also switches the examples to use human sizes instead of a number of bytes (since we actually support that already). That should decrease the likelihood of mistakes for those values, and make it very easy for an admin to see at a glance if they might've overdone it.

So far, there's no evidence of people misconfiguring these values, let alone it being common place. I think for now we'll refrain from adding additional warnings, as it's hard to determine what the "safe" values are and when servers are operating in closed federation much bigger values might be perfectly fine. We have suggestions for each of these values in our documentation. If admins chose to ignore them they should be able to do so without GtS warning about them on startup.

[daenney]

I'm going to close this one for now. If people do run into accidental misconfiguration issues please do add to this issue and we'll reconsider if we need to do something more about it.