Backward compatibility tests on states in DB

[infwinston]

In the latest storage PR we renamed some states of buckets (e.g., DONE to READY)
https://github.com/sky-proj/sky/pull/658/files#diff-db4566a60e392e4076a1a2bc4eaa56e57c4c2a060b23ff98e8b7b71e3e3d35f1L131

This caused a backward compatibility issue to sky storage ls to the buckets I created earlier and I had to manually clean up storage table in DB and remove the buckets myself.

(sky) weichiang@blaze:~/repos$ sky storage ls              
Traceback (most recent call last):                                   
  File "/data/weichiang/miniconda3/envs/sky/bin/sky", line 33, in <module> 
    sys.exit(load_entry_point('sky', 'console_scripts', 'sky')())        
  File "/data/weichiang/miniconda3/envs/sky/lib/python3.8/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)                      
  File "/data/weichiang/miniconda3/envs/sky/lib/python3.8/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)                                       
  File "/data/weichiang/miniconda3/envs/sky/lib/python3.8/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))    
  File "/data/weichiang/miniconda3/envs/sky/lib/python3.8/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/data/weichiang/miniconda3/envs/sky/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/data/weichiang/miniconda3/envs/sky/lib/python3.8/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/home/eecs/weichiang/repos/sky/sky/cli.py", line 1846, in storage_ls
    storage_stat = global_user_state.get_storage()
  File "/home/eecs/weichiang/repos/sky/sky/global_user_state.py", line 371, in get_storage
    'status': StorageStatus[status],
  File "/data/weichiang/miniconda3/envs/sky/lib/python3.8/enum.py", line 387, in __getitem__
    return cls._member_map_[name]
KeyError: 'DONE'

Should we add backward compatibility tests on states (of cluster/storage) stored in DB to avoid users hitting such errors after upgrading Sky?
One simple test would be to use previous version of Sky to run smoke tests and later update Sky. All sky commands still need to function normally.

[Michaelvll]

Yes, a test should be nice. For the backward compatibility, I added a way to keep class backward compatible by __setstate__ and versioning in #714.

[Michaelvll]

This issue seems to be staled, as people should have already upgraded to the latest skypilot. Closing this issue for now. Please feel free to re-open it if the problem persists.