json parse error when calling `artist` method #452

aome510 · 2024-01-06T11:48:27Z

Describe the bug
Client now returns

Bad request: json parse error: invalid type: floating point `49022656`, expected u32 at line 1 column 130: invalid type: floating point `49022656`, expected u32 at line 1 column 130

for every artist request. The reason seems to be because Spotify changes the type of followers.total and other fields to float.

To Reproduce
Steps to reproduce the behavior:

Run

curl --request GET \
        --url https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d \
        --header 'Authorization: Bearer $TOKEN'

Observe that follower.total is not an integer. Example result:

{"external_urls":{"spotify":"https://open.spotify.com/artist/1dfeR4HaWDbWqFHLkxsg1d"},"followers":{"href":null,"total":4.9022656E7},"genres":["classic rock","glam rock","rock"],"href":"https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d","id":"1dfeR4HaWDbWqFHLkxsg1d","images":[{"url":"https://i.scdn.co/image/b040846ceba13c3e9c125d68389491094e7f2982","height":806.0,"width":999.0},{"url":"https://i.scdn.co/image/af2b8e57f6d7b5d43a616bd1e27ba552cd8bfd42","height":516.0,"width":640.0},{"url":"https://i.scdn.co/image/c06971e9ff81696699b829484e3be165f4e64368","height":161.0,"width":200.0},{"url":"https://i.scdn.co/image/6dd0ffd270903d1884edf9058c49f58b03db893d","height":52.0,"width":64.0}],"name":"Queen","popularity":84.0,"type":"artist","uri":"spotify:artist:1dfeR4HaWDbWqFHLkxsg1d"}

Notice that some fields' type are changed to float such as followers.total, popularity

The text was updated successfully, but these errors were encountered:

ramsayleung · 2024-01-06T16:28:47Z

It's weird that the total and popularity are marked as integer in their official document, but actually return float data.

I tried to reproduce this bug by following your instruction in the document console, but the response is different from yours.

curl --request GET \
  --url https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d \
  --header 'Authorization: Bearer $TOKEN'

{
  "external_urls": {
    "spotify": "https://open.spotify.com/artist/1dfeR4HaWDbWqFHLkxsg1d"
  },
  "followers": {
    "href": null,
    "total": 49022656
  },
  "genres": ["classic rock", "glam rock", "rock"],
  "href": "https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d?locale=en-US%2Cen%3Bq%3D0.9%2Czh-CN%3Bq%3D0.8%2Czh%3Bq%3D0.7",
  "id": "1dfeR4HaWDbWqFHLkxsg1d",
  "images": [
    {
      "url": "https://i.scdn.co/image/b040846ceba13c3e9c125d68389491094e7f2982",
      "height": 806,
      "width": 999
    },
    {
      "url": "https://i.scdn.co/image/af2b8e57f6d7b5d43a616bd1e27ba552cd8bfd42",
      "height": 516,
      "width": 640
    },
    {
      "url": "https://i.scdn.co/image/c06971e9ff81696699b829484e3be165f4e64368",
      "height": 161,
      "width": 200
    },
    {
      "url": "https://i.scdn.co/image/6dd0ffd270903d1884edf9058c49f58b03db893d",
      "height": 52,
      "width": 64
    }
  ],
  "name": "Queen",
  "popularity": 84,
  "type": "artist",
  "uri": "spotify:artist:1dfeR4HaWDbWqFHLkxsg1d"
}

followers.total, popularity are both integer rather than float.

KaydaFox · 2024-01-06T17:09:40Z

im getting this same error too, think spotify may have changed something because this was all fine yesterday
response from curl:

{
	"external_urls": {
		"spotify": "https://open.spotify.com/artist/4bwIf0yXJf0F9AmOl2J78M"
	},
	"followers": {
		"href": null,
		"total": 194172.0
	},
	"genres": [
		"denpa-kei",
		"rhythm game"
	],
	"href": "https://api.spotify.com/v1/artists/4bwIf0yXJf0F9AmOl2J78M",
	"id": "4bwIf0yXJf0F9AmOl2J78M",
	"images": [
		{
			"url": "https://i.scdn.co/image/ab6761610000e5ebc1f9dd30f82038e91e865d82",
			"height": 640.0,
			"width": 640.0
		},
		{
			"url": "https://i.scdn.co/image/ab67616100005174c1f9dd30f82038e91e865d82",
			"height": 320.0,
			"width": 320.0
		},
		{
			"url": "https://i.scdn.co/image/ab6761610000f178c1f9dd30f82038e91e865d82",
			"height": 160.0,
			"width": 160.0
		}
	],
	"name": "Camellia",
	"popularity": 51.0,
	"type": "artist",
	"uri": "spotify:artist:4bwIf0yXJf0F9AmOl2J78M"
}```

ramsayleung · 2024-01-06T20:36:50Z

Yes, I can reproduce your problem now, I've submitted a ticket to Spotify to ask for clarity: https://community.spotify.com/t5/Spotify-for-Developers/The-response-type-of-artist-endpoint-is-marked-as-Integer-in/m-p/5800044

gansheer · 2024-01-06T20:59:18Z

Hi @ramsayleung, just so you know there is an error in the issue link https://github.com/ramsayleung/rspotify/issues/452m (trailing m) in your Spotify ticket.

ramsayleung · 2024-01-06T22:02:49Z

Thanks for your heads-up, I've revised the link to the correct one. But it's odd that a developer replied in the ticket saying what he got with curl was integer data instead of float data.

Unfortunately, this ticket is marked as spam...

I've submitted an abuse report.

PMARZV · 2024-01-06T23:08:14Z

Hey here! Developing an app for android and i have the same problem with the GETARTISTS endpoint

aome510 · 2024-01-20T19:37:08Z

I don't think Spotify will give their response anytime soon. I wonder if we have any plans to mitigate this issue @ramsayleung ?

ramsayleung · 2024-01-23T03:25:14Z

One of the workarounds comes to my mind is converting the Integer type to float type to bypass the issue. But in my perspective, I don't think it's a great idea because

The SDK shouldn't silently swallow the bug caused by the Spotify server side, a better solution will be prompting the issue to Spotify and push them to fix the problem
It will introduce a breaking change in this library, which's definitely unnecessary and avoidable.

I personally prefer to keep escalating and making "noise" to the Spotify developers, the louder noise we could make, the higher probability the Spotify team could be aware of this problem and fix it.

ramsayleung · 2024-01-23T03:49:17Z

I've sent a message to escalate this issue to the Spotify team via this link: https://support.spotify.com/ca-en/contact-spotify-support/, here is the response from their associate:

I already forward this issue to the team backstage. Also, the tech folks backstage confirm that they're already ahead of this and currently working for a fix. No need to worry now :)

Also, our Tech folks are known for their hard work so this will be fixed in no time! Consider this issue resolved.

aome510 · 2024-01-23T03:53:16Z

I've sent a message to escalate this issue to the Spotify team via this link: https://support.spotify.com/ca-en/contact-spotify-support/, here is the response from their associate:

I already forward this issue to the team backstage. Also, the tech folks backstage confirm that they're already ahead of this and currently working for a fix. No need to worry now :)

Also, our Tech folks are known for their hard work so this will be fixed in no time! Consider this issue resolved.

Great! Thank you @ramsayleung

grantshandy · 2024-01-29T17:08:50Z

This issue is also holding up the development of a project of mine. If it isn't fixed soon by the spotify developers would rspotify support deserializing all u32 fields with a custom Deserializer a la #[serde(deserialize_with = "as_u32")]? as_u32 would automatically convert float fields into int ones and keep the same default behavior as before. It's pretty hacky, but it would fix the issue for now/if it arises again (im feeling pretty pessimistic lol). It'd be a less permissive version of serde-this-or-that's as_u64.

This solution wouldn't introduce breaking changes in the API as @ramsayleung pointed out, and the fix would continue to work if spotify does get around to fixing the issue. Plus, it would only have the overhead of converting any given f64 to a u32. If there's any interest in this sort of fix, let me know and I can open up a PR pretty quick.

KaydaFox · 2024-01-29T17:47:04Z

This issue is also holding up the development of a project of mine. If it isn't fixed soon by the spotify developers would rspotify support deserializing all u32 fields with a custom Deserializer a la #[serde(deserialize_with = "as_u32")]? as_u32 would automatically convert float fields into int ones and keep the same default behavior as before. It's pretty hacky, but it would fix the issue for now/if it arises again (im feeling pretty pessimistic lol). It'd be a less permissive version of serde-this-or-that's as_u64.

This solution wouldn't introduce breaking changes in the API as @ramsayleung pointed out, and the fix would continue to work if spotify does get around to fixing the issue. Plus, it would only have the overhead of converting any given f64 to a u32. If there's any interest in this sort of fix, let me know and I can open up a PR pretty quick.

One thing you could do as a temporary workaround is this

        match rspotify
            .api_get(&format!("artists/{}", artist_id), &Query::new())
            .await
        {
            Ok(artist) => {
                // Manually parse the json for now
                let artist_json: Result<Value, _> = serde_json::from_str(&artist);
                match artist_json {
                }
        }
        ```
        im not sure if its the bestt way but its what im using for now and it seems to work alright, still learning stuff lol
        but it stopped me from needing to halt development on my stuff temporarily

ramsayleung · 2024-01-30T03:18:51Z

One thing you could do as a temporary workaround is this

But this temporary workaround only for the you, the sole developer who decides to engage this approach.

I would try to implement the as_u32 approach without imposing breaking change as a short term solution, as we actually don't know when will Spotify fix this issue.

grantshandy · 2024-01-30T04:09:00Z

Should I contribute the fix? I can do it in the next two days unless you'd like to implement it yourself.

ramsayleung · 2024-01-30T04:13:58Z

I could provide the code snippet for your reference, feel free to contribute and issue a PR :)

pub mod data_type_patcher {
    use serde::{Deserialize, Deserializer};
    pub fn as_u32<'de, D>(deserializer: D) -> Result<u32, D::Error>
where
    D: Deserializer<'de>,
{
    let float_data: f64 = Deserialize::deserialize(deserializer)?;

    let u32_data = float_data as u32;

    Ok(u32_data)
}
}

m-torhan · 2024-02-01T09:53:22Z

As I didn't see any PR related to this I let myself to implement it #457.

I tested it with https://github.com/aome510/spotify-player and it worked well.

Due to Spotify API returning floats instead of uints for some fields of the Artist it cannot be parsed. This commit adds deserializing float values into u32 as a workaround. Related issue: [issue](ramsayleung#452) Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>

aome510 · 2024-02-03T17:34:39Z

Look like a workaround patch landed to master. @ramsayleung do you have any plans to release a new version?

ramsayleung · 2024-02-09T02:53:30Z

I am on vacation right now, I will release a new version when I am back :)

…

On Sun, Feb 4, 2024 at 1:34 AM Thang Pham ***@***.***> wrote: Look like a workaround patch landed to master. @ramsayleung <https://github.com/ramsayleung> do you have any plans to release a new version? — Reply to this email directly, view it on GitHub <#452 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADMGCSMC32AKD5FDO26MFZ3YRZYLXAVCNFSM6AAAAABBPP63O2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVGQYDQMRVGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

aome510 · 2024-03-02T22:58:02Z

I'm now getting the same error but with get-several-tracks API call. To reproduce, run

curl --request GET \
         --url "https://api.spotify.com/v1/tracks?ids=5O2P9iiztwhomNh8xkR9lJ" \
         --header 'Authorization: Bearer $TOKEN'

Observe that many fields expecting integer return float. The response that I got

{"tracks":[{"album":{"album_type":"album","artists":[{"external_urls":{"spotify":"https://open.spotify.com/artist/4AK6F7OLvEQ5QYCBNiQWHq"},"href":"https://api.spotify.com/v1/artists/4AK6F7OLvEQ5QYCBNiQWHq","id":"4AK6F7OLvEQ5QYCBNiQWHq","name":"One Direction","type":"artist","uri":"spotify:artist:4AK6F7OLvEQ5QYCBNiQWHq"}],"available_markets":["AR","AU","AT","BE","BO","BR","BG","CA","CL","CO","CR","CY","CZ","DK","DO","DE","EC","EE","SV","FI","FR","GR","GT","HN","HK","HU","IS","IE","IT","LV","LT","LU","MY","MT","MX","NL","NZ","NI","NO","PA","PY","PE","PH","PL","PT","SG","SK","ES","SE","CH","TW","TR","UY","US","GB","AD","LI","MC","ID","JP","TH","VN","RO","IL","ZA","SA","AE","BH","QA","OM","KW","EG","MA","DZ","TN","LB","JO","PS","IN","BY","KZ","MD","UA","AL","BA","HR","ME","MK","RS","SI","KR","BD","PK","LK","GH","KE","NG","TZ","UG","AG","AM","BS","BB","BZ","BT","BW","BF","CV","CW","DM","FJ","GM","GE","GD","GW","GY","HT","JM","KI","LS","LR","MW","MV","ML","MH","FM","NA","NR","NE","PW","PG","WS","SM","ST","SN","SC","SL","SB","KN","LC","VC","SR","TL","TO","TT","TV","VU","AZ","BN","BI","KH","CM","TD","KM","GQ","SZ","GA","GN","KG","LA","MO","MR","MN","NP","RW","TG","UZ","ZW","BJ","MG","MU","MZ","AO","CI","DJ","ZM","CD","CG","IQ","LY","TJ","VE","ET","XK"],"external_urls":{"spotify":"https://open.spotify.com/album/4gCNyS7pidfK3rKWhB3JOY"},"href":"https://api.spotify.com/v1/albums/4gCNyS7pidfK3rKWhB3JOY","id":"4gCNyS7pidfK3rKWhB3JOY","images":[{"url":"https://i.scdn.co/image/ab67616d0000b273d304ba2d71de306812eebaf4","width":640.0,"height":640.0},{"url":"https://i.scdn.co/image/ab67616d00001e02d304ba2d71de306812eebaf4","width":300.0,"height":300.0},{"url":"https://i.scdn.co/image/ab67616d00004851d304ba2d71de306812eebaf4","width":64.0,"height":64.0}],"is_playable":true,"name":"FOUR (Deluxe)","release_date":"2014-11-17","release_date_precision":"day","total_tracks":16.0,"type":"album","uri":"spotify:album:4gCNyS7pidfK3rKWhB3JOY"},"artists":[{"external_urls":{"spotify":"https://open.spotify.com/artist/4AK6F7OLvEQ5QYCBNiQWHq"},"href":"https://api.spotify.com/v1/artists/4AK6F7OLvEQ5QYCBNiQWHq","id":"4AK6F7OLvEQ5QYCBNiQWHq","name":"One Direction","type":"artist","uri":"spotify:artist:4AK6F7OLvEQ5QYCBNiQWHq"}],"available_markets":["AR","AU","AT","BE","BO","BR","BG","CA","CL","CO","CR","CY","CZ","DK","DO","DE","EC","EE","SV","FI","FR","GR","GT","HN","HK","HU","IS","IE","IT","LV","LT","LU","MY","MT","MX","NL","NZ","NI","NO","PA","PY","PE","PH","PL","PT","SG","SK","ES","SE","CH","TW","TR","UY","US","GB","AD","LI","MC","ID","JP","TH","VN","RO","IL","ZA","SA","AE","BH","QA","OM","KW","EG","MA","DZ","TN","LB","JO","PS","IN","BY","KZ","MD","UA","AL","BA","HR","ME","MK","RS","SI","KR","BD","PK","LK","GH","KE","NG","TZ","UG","AG","AM","BS","BB","BZ","BT","BW","BF","CV","CW","DM","FJ","GM","GE","GD","GW","GY","HT","JM","KI","LS","LR","MW","MV","ML","MH","FM","NA","NR","NE","PW","PG","WS","SM","ST","SN","SC","SL","SB","KN","LC","VC","SR","TL","TO","TT","TV","VU","AZ","BN","BI","KH","CM","TD","KM","GQ","SZ","GA","GN","KG","LA","MO","MR","MN","NP","RW","TG","UZ","ZW","BJ","MG","MU","MZ","AO","CI","DJ","ZM","CD","CG","IQ","LY","TJ","VE","ET","XK"],"disc_number":1.0,"duration_ms":226600.0,"explicit":false,"external_ids":{"isrc":"GBHMU1400165"},"external_urls":{"spotify":"https://open.spotify.com/track/5O2P9iiztwhomNh8xkR9lJ"},"href":"https://api.spotify.com/v1/tracks/5O2P9iiztwhomNh8xkR9lJ","id":"5O2P9iiztwhomNh8xkR9lJ","is_local":false,"is_playable":true,"name":"Night Changes","popularity":89.0,"preview_url":"https://p.scdn.co/mp3-preview/359be833b46b250c696bbb64caa5dc91f2a38c6a?cid=b28a574d8ad44fc7b5a89994431a8f88","track_number":7.0,"type":"track","uri":"spotify:track:5O2P9iiztwhomNh8xkR9lJ"}]}

Not sure if the above issue is only reproducible on my end because the same request on https://developer.spotify.com/documentation/web-api/reference/get-several-tracks still returns the correct type.

Found this issue when trying to investigate aome510/spotify-player#383

ramsayleung · 2024-03-02T23:33:24Z

I have to say, it's "scope creep" of Spotify's bug, other developers have reported the same observation in other endpoints, it becomes less possible to be patched and bypassed individually by the community-driven library, this problem has to be fixed by the Spotify itself.

aome510 · 2024-03-02T23:40:55Z

I have to say, it's "scope creep" of Spotify's bug, other developers have reported the same observation in other endpoints, it becomes less possible to be patched and bypassed individually by the community-driven library, this problem has to be fixed by the Spotify itself.

Yeah, I agree. At the same time, it's kinda frustrating to see that Spotify doesn't seem to take any actions.

ramsayleung · 2024-03-03T04:01:43Z

The new version of RSpotify would like to pend for a while for the response from Spotify, since it's meanless to release a new version while Spotify keeps breaking their endpoints, the community is unable to catch up to release new patch.

Resolves #330 Resolves #383 Resolves #384 Related upstream issues: - ramsayleung/rspotify#452 - ramsayleung/rspotify#459 This PR implements patches for upstream Spotify API bugs by using `client::Client::internal_call` to get Spotify data instead of `rspotify` APIs. Using `internal_call` allows the app's client to avoid parsing errors when `rspotify` parses Spotify API responses and fixes - artist page not loading - recommendation/radio tracks page not loading - playlist page not loading for playlists with no image

ed9w2in6 · 2024-03-07T07:05:47Z

@aome510 @ramsayleung
Seems to be fixed ^_^

As expected this is caused by some library that they are using.

I am using spotify_player today and found that at the the artist endpoint works now.

ramsayleung · 2024-03-08T19:16:19Z

Closing since this issue has been resolved by Spotify.

aome510 added bug Something isn't working help wanted Extra attention is needed labels Jan 6, 2024

aome510 mentioned this issue Jan 6, 2024

Artist page/widget not loading aome510/spotify-player#330

Closed

Marekkon5 added a commit to Marekkon5/rspotify that referenced this issue Jan 16, 2024

Temp fix for ramsayleung#452

c26e5a6

Marekkon5 mentioned this issue Jan 16, 2024

Genres not being overwritten Marekkon5/onetagger#319

Open

DavidHospital mentioned this issue Jan 25, 2024

Can't Access Artist Page Rigellute/spotify-tui#1116

Closed

m-torhan mentioned this issue Feb 1, 2024

Workaround for Spotify API bug #457

Merged

4 tasks

aome510 mentioned this issue Mar 3, 2024

Patch spotify API bugs aome510/spotify-player#386

Merged

This was referenced Mar 8, 2024

Revert the patch for Spotify API Bug after Spotify fixed the problem. #461

Merged

Upgrade RSpotify to 0.13 #462

Merged

ramsayleung closed this as completed Mar 8, 2024

aome510 mentioned this issue Mar 22, 2024

Playlists don't show up on the homepage aome510/spotify-player#398

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

json parse error when calling `artist` method #452

json parse error when calling `artist` method #452

aome510 commented Jan 6, 2024 •

edited

Loading

ramsayleung commented Jan 6, 2024 •

edited

Loading

KaydaFox commented Jan 6, 2024

ramsayleung commented Jan 6, 2024

gansheer commented Jan 6, 2024

ramsayleung commented Jan 6, 2024 •

edited

Loading

PMARZV commented Jan 6, 2024

aome510 commented Jan 20, 2024

ramsayleung commented Jan 23, 2024 •

edited

Loading

ramsayleung commented Jan 23, 2024

aome510 commented Jan 23, 2024

grantshandy commented Jan 29, 2024

KaydaFox commented Jan 29, 2024 •

edited

Loading

ramsayleung commented Jan 30, 2024

grantshandy commented Jan 30, 2024

ramsayleung commented Jan 30, 2024 •

edited

Loading

m-torhan commented Feb 1, 2024

aome510 commented Feb 3, 2024

ramsayleung commented Feb 9, 2024 via email

aome510 commented Mar 2, 2024 •

edited

Loading

ramsayleung commented Mar 2, 2024

aome510 commented Mar 2, 2024

ramsayleung commented Mar 3, 2024

ed9w2in6 commented Mar 7, 2024

ramsayleung commented Mar 8, 2024

json parse error when calling artist method #452

json parse error when calling artist method #452

Comments

aome510 commented Jan 6, 2024 • edited Loading

ramsayleung commented Jan 6, 2024 • edited Loading

KaydaFox commented Jan 6, 2024

ramsayleung commented Jan 6, 2024

gansheer commented Jan 6, 2024

ramsayleung commented Jan 6, 2024 • edited Loading

PMARZV commented Jan 6, 2024

aome510 commented Jan 20, 2024

ramsayleung commented Jan 23, 2024 • edited Loading

ramsayleung commented Jan 23, 2024

aome510 commented Jan 23, 2024

grantshandy commented Jan 29, 2024

KaydaFox commented Jan 29, 2024 • edited Loading

ramsayleung commented Jan 30, 2024

grantshandy commented Jan 30, 2024

ramsayleung commented Jan 30, 2024 • edited Loading

m-torhan commented Feb 1, 2024

aome510 commented Feb 3, 2024

ramsayleung commented Feb 9, 2024 via email

aome510 commented Mar 2, 2024 • edited Loading

ramsayleung commented Mar 2, 2024

aome510 commented Mar 2, 2024

ramsayleung commented Mar 3, 2024

ed9w2in6 commented Mar 7, 2024

ramsayleung commented Mar 8, 2024

json parse error when calling `artist` method #452

json parse error when calling `artist` method #452

aome510 commented Jan 6, 2024 •

edited

Loading

ramsayleung commented Jan 6, 2024 •

edited

Loading

ramsayleung commented Jan 6, 2024 •

edited

Loading

ramsayleung commented Jan 23, 2024 •

edited

Loading

KaydaFox commented Jan 29, 2024 •

edited

Loading

ramsayleung commented Jan 30, 2024 •

edited

Loading

aome510 commented Mar 2, 2024 •

edited

Loading