Skip to content

Commit

Permalink
first commit for RedLock
Browse files Browse the repository at this point in the history
  • Loading branch information
sewenew committed Oct 30, 2019
1 parent 3ba84a8 commit 856ed96
Show file tree
Hide file tree
Showing 2 changed files with 221 additions and 0 deletions.
112 changes: 112 additions & 0 deletions src/sw/redis++/recipes/redlock.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
/**************************************************************************
Copyright (c) 2017 sewenew
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*************************************************************************/

#include "redlock.h"

namespace sw {

namespace redis {

std::chrono::milliseconds RedMutex::try_lock(const std::string &val, const std::chrono::milliseconds &ttl) {
auto start = std::chrono::steady_clock::now();

if (!_redis.set(_resource, val, ttl, UpdateType::NOT_EXIST)) {

This comment has been minimized.

Copy link
@wingunder

wingunder Oct 30, 2019

Contributor

Are your _resource and val variables not swapped?
You have to lock with a key name, that the user supplies (val in your case), as he/she can choose a non-used key, based on the knowledge of their key namespace.
The key has to be set to a random value (_resource in your case), which you should generate every time you lock.

This comment has been minimized.

Copy link
@sewenew

sewenew Oct 31, 2019

Author Owner

NO. _resource is the key name, and val is the random string. When user want to use try_lock, he/she must specify a resource key that he/she wants to lock.

This comment has been minimized.

Copy link
@wingunder

wingunder Oct 31, 2019

Contributor

Ok, sorry, I was fooled by the fact that you made a random key (_resource).
So if I get it right now, you're not allowing the user to specify a 'lockKey'?
If this is right, we'll run the (very small) risk of keys clashing, possibly causing a lock to never be able to be acquired.
I'd add a key parameter. Expecting from a user to supply a lockKeyName, is really not asking too much. A user can supply anything. It doesn't have to be a random key name, just a key name that is not in use.

This comment has been minimized.

Copy link
@sewenew

sewenew Oct 31, 2019

Author Owner

If I understand correctly, when you say lockKey, it means the random value, i.e. the my_random_value in the RedLock algorithm doc?

Yes, user doesn't need to specify the random value. Instead, I'll create a random string for the user. Use a similar method mentioned in the Redlock algorithm doc:

For example a safe pick is to seed RC4 with /dev/urandom, and generate a pseudo random stream from that.

I use the /dev/random to generate a seed, and use that seed to create a 20 bytes long string. /dev/random will generate a real random number, NOT pseudo-random. So it's won't clashing.

This comment has been minimized.

Copy link
@sewenew

sewenew Oct 31, 2019

Author Owner

It seems that's also how other implementation of RedLock generate the random value.

throw Error("failed to lock " + _resource);
}

auto stop = std::chrono::steady_clock::now();
auto elapse = stop - start;

This comment has been minimized.

Copy link
@wingunder

wingunder Oct 30, 2019

Contributor

This times a 'SET' command and it's round trip to the Redis server.
Unless there's some serious problem with the Redis setup, this takes a few microseconds.
On my PC it takes an average of 8.6uSec.
Try this: $ redis-benchmark -t set -n 100000 -q
In case of a connection problem, set() will probably either return false or throw an exception, so elapse will practically always be == std::chrono::milliseconds(0).

This comment has been minimized.

Copy link
@sewenew

sewenew Oct 31, 2019

Author Owner

I think you were sending the command to a local Redis server. However, if sending a command to a remote server, normally the round trip time should be several milliseconds or even longer. Especially, when we implement a more robust RedLock with Redis Cluster, we need to lock on multiple Redis nodes, and that will cost much more time.


auto time_left = std::chrono::duration_cast<std::chrono::milliseconds>(ttl - elapse);

if (time_left < std::chrono::milliseconds(0)) {
// No time left for the lock.
try {
unlock(_resource);

This comment has been minimized.

Copy link
@wingunder

wingunder Oct 30, 2019

Contributor

The lock has timed out, if you get here.
If the lock timed out, it's gone inside Redis, so there's no need to release it inside Redis.
Redis has a latency of 0-1ms for expiring keys (see here), so this won't get Redis to release it faster, if that was the goal.

This comment has been minimized.

Copy link
@sewenew

sewenew Oct 31, 2019

Author Owner

Yes, I want to release the lock ASAP. Imagine if we try to lock for 10 milliseconds, however, the round trip time costs 20 milliseconds. So when we get the response, we should not hold the lock any longer. But the other processes still cannot lock the resource, since the resource key haven't expired.

Since this key will expire anyway, it should not be a problem even if we don't unlock it manually. However, release the lock ASAP is always a good idea :)

I also checked the redlock-rb implementation by @antirez (thanks for your link on other implementations ^_^), he also tried to release the lock ASAP. You can take a look at here.

} catch (const Error &err) {
throw Error("failed to lock " + _resource);
}
}

return time_left;
}

bool RedMutex::try_lock(const std::string &val,
const std::chrono::time_point<std::chrono::system_clock> &tp) {
try {
try_lock(val, _ttl(tp));
} catch (const Error &err) {
return false;
}

return true;
}

bool RedMutex::extend_lock(const std::string &val,
const std::chrono::time_point<std::chrono::system_clock> &tp) {
auto tx = _redis.transaction(true);
auto r = tx.redis();
try {
auto ttl = _ttl(tp);

r.watch(_resource);

auto id = r.get(_resource);
if (id && *id == val) {
auto reply = tx.pexpire(_resource, ttl).exec();
if (!reply.get<bool>(0)) {
throw Error("this should not happen");
}
}
} catch (const Error &err) {
// key has been modified or other error happens, failed to extend the lock.
return false;
}

return true;
}

void RedMutex::unlock(const std::string &val) {
auto tx = _redis.transaction(true);
auto r = tx.redis();
try {
r.watch(_resource);

auto id = r.get(_resource);
if (id && *id == val) {
auto reply = tx.del(_resource).exec();
if (reply.get<long long>(0) != 1) {
throw Error("this should not happen");
}
}
} catch(const WatchError &err) {
// key has been modified. Do nothing, just let it go.
}
}

std::chrono::milliseconds RedMutex::_ttl(const SysTime &tp) const {
auto cur = std::chrono::system_clock::now();
auto ttl = tp - cur;
if (ttl.count() < 0) {
throw Error("time already pasts");
}

return std::chrono::duration_cast<std::chrono::milliseconds>(ttl);
}

}

}
109 changes: 109 additions & 0 deletions src/sw/redis++/recipes/redlock.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
/**************************************************************************
Copyright (c) 2017 sewenew
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*************************************************************************/

#ifndef SEWENEW_REDISPLUSPLUS_RECIPES_REDLOCK_H
#define SEWENEW_REDISPLUSPLUS_RECIPES_REDLOCK_H

#include <cassert>
#include <random>
#include <chrono>
#include <string>
//#include "../redis++.h"
#include <sw/redis++/redis++.h>

namespace sw {

namespace redis {

class RedMutex {
public:
RedMutex(Redis &redis, const std::string &resource) : _redis(redis), _resource(resource) {}

std::chrono::milliseconds try_lock(const std::string &val, const std::chrono::milliseconds &ttl);

bool try_lock(const std::string &val,
const std::chrono::time_point<std::chrono::system_clock> &tp);

bool extend_lock(const std::string &val,
const std::chrono::time_point<std::chrono::system_clock> &tp);

void unlock(const std::string &val);

private:
using SysTime = std::chrono::time_point<std::chrono::system_clock>;

std::chrono::milliseconds _ttl(const SysTime &tp) const;

Redis &_redis;

std::string _resource;
};

template <typename Mutex>
class RedLock {
public:
RedLock(Mutex &mut, std::defer_lock_t) : _mut(mut), _lock_val(_lock_id()) {}

~RedLock() {
if (_owned) {
unlock();
}
}

std::chrono::milliseconds try_lock(const std::chrono::milliseconds &ttl) {
return _mut.try_lock(_lock_val, ttl);
}

void unlock() {
_mut.unlock(_lock_val);
}

private:
std::string _lock_id() {
std::random_device dev;
std::mt19937 random_gen(dev());
int range = 10 + 26 + 26 - 1;
std::uniform_int_distribution<> dist(0, range);
std::string id;
id.reserve(20);
for (int i = 0; i != 20; ++i) {
auto idx = dist(random_gen);
if (idx < 10) {
id.push_back('0' + idx);
} else if (idx < 10 + 26) {
id.push_back('a' + idx - 10);
} else if (idx < 10 + 26 + 26) {
id.push_back('A' + idx - 10 - 26);
} else {
assert(false);
}
}

return id;
}

Mutex &_mut;

bool _owned = false;

std::string _lock_val;
};

}

}

#endif // end SEWENEW_REDISPLUSPLUS_RECIPES_REDLOCK_H

0 comments on commit 856ed96

Please sign in to comment.