Java World Knowledge Sharing Portal: Redis NoSQL schemaless Mutlidata type supporting Scalable, durable , recoverable Database

Redis is nosql database. It is also called as Remote dictionary server. Generally developer use the redis to use it as an intermediate between the application and original DB for cache first mechanism. But Redis itself is a memory first Key value type no sql data base. It support different type of data type like string, set, hash etc.
Let's take an example of social app. lets assume we had an requirement where we need to store the user information like name, add, telephone number, password etc. as a social app each use is connected to other user so we have graph DB like requirement. Additionally we also need to have quick search on this huge db like apache solr. Also this app has facilities to store the files i.e. document voice, video, txt file etc. All this can be done in one database and that is Redis. Redis come with different module. The core one is called Redis core which support all above datatype like string set has etc but it also comes with separate module to address issues like graph db, quick search, storing files etc. for this we have the following modules

1- Quick search – Redis search
2- Storing user informaiton – Redis time series DB.
3- Strong audio, video and files – Redis JSON
4- Graph DB- Redis graph.

SCALLING:-

In addition to this Redis has one big advantage of easy SCALLING in comparison of other nosql DB. It support scalling using two option

1- Clustering
2- Sharding

1- Clustering – In this we replicas the master Redis DB so that Master work as read-write db and rest other work as read only db. when master dies any one of the replica become master. In addition to this redis also support best option of back up. It has two options

a:- Timestamp :- in this concept we take the redis db back up at definite interval of time. only issues is if the last timestamp back up fail we get the date till the secondlast timestamp for recovery.
b:- AOF – append only file – in this we append the changes first in the file (backup) and then do the changes in redis. here we had runtime back up and no data is loss but this has some performance issues.

2-Sharding :- in this concept we divide our master data into different data set and store the data set itself on different machine or Node. This will also scale the machine also give us the data disaster recovery mechanism.

In redis We can use the both concept at the same time. To achieve this, modifies the redis.conf file and modify “appendonly” parameter from “no” to “yes”

To take the time stamp just execute the save option on the command prompt of redis cli tool it will create dump.rdb as shown below.

Back up:-

Now the question is how redis support the 100% back up. lets say if not only the node but the complete serveror M/C is down in that case we are using the remote storing concept i.e. if you are using Cloud the we will have Redis service and DB running EC2 and their back up is taken on Elastic block storage server. so in this case if the whole EC2 is down still we have the back up on Elastic.

Durability:-

Redis also support Durability :- for this it use the concept of Redis on flash. i.e. it will also use the SSD card to store the data in ram. Most frequent data will be stored in RAM and less will be stored in SSD.

There is also a concept called as active active geo distribution. Using this concept we can restore the whole data center. let say we have a data center in india and we have customer in UK and Canada for their speedy application use it is better to have the data nearer to them so taht latency will be less. This is achieved by having data center in canada and also in uk. but the issues is how to sync them all. this is also taken care by redis enterprise. so if one whole datacenter fail we can build it from taking the record from other datacenter

Now the question is in world of Kubernetes how to manage this Redis for scalability, storing, backup , durability etc. for this We have an Redis operator already build which take care of all this things i.e. deployment, configuring, scaling , backup, recovering and much more.

Now lets try to install Redis as we are using windows please follow below steps

Go to belwo sites and clone the repo.

https://github.com/zkteco-home/redis-windows

Install service instance:

redis-server –service-install redis.conf –loglevel verbose

Staring service
redis-server –service-start

Starting client

C:\redis-windows>redis-server --service-install redis.conf --loglevel verbose
[9192] 17 Jan 13:16:35.992 # Granting read/write access to 'NT AUTHORITY\NetworkService' on: "C:\redis-windows" "C:\redis-windows" "C:\redis-windows\"
[9192] 17 Jan 13:16:35.993 # Redis successfully installed as a service.
 
C:\redis-windows>redis-server --service-start
[7744] 17 Jan 13:17:11.154 # Redis service successfully started.
 
C:\redis-windows>redis-cli.exe
127.0.0.1:6379>
 
check connection using word ping we get pong
 
'-  checking connection is live
 
C:\redis-windows>redis-cli.exe
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> echo "hello world"
"hello world"
 
'- Quitting connection
127.0.0.1:6379> quit
 
C:\redis-windows>
 
 
Now lets play with few of the basic concept of Redis.
 
'- setting values
 
127.0.0.1:6379> set foo siddhu
OK
127.0.0.1:6379> get foo
"siddhu"
 
'- incr and decr values.
 
 
127.0.0.1:6379> incr siddhu
(integer) 1
127.0.0.1:6379> incr siddhu
(integer) 2
127.0.0.1:6379> incr siddhu
(integer) 3
127.0.0.1:6379> get siddhu
"3"
127.0.0.1:6379> decr siddhu
(integer) 2
127.0.0.1:6379> decr siddhu
(integer) 1
127.0.0.1:6379> decr siddhu
(integer) 0
127.0.0.1:6379> decr siddhu
(integer) -1
127.0.0.1:6379>
 
'- To check if exist 1 means key exist and 0 means it does not exist i.e. we did not set the value of that key.
 
127.0.0.1:6379> exists foo
(integer) 1
127.0.0.1:6379> exists siddhu
(integer) 1
127.0.0.1:6379> exists test
(integer) 0
127.0.0.1:6379>
 
'- to delete the key use belwo command 1 means key is deleted and 0 means did not found the key.
 
127.0.0.1:6379>  del siddhu
(integer) 1
127.0.0.1:6379> del siddhu
(integer) 0
127.0.0.1:6379> get siddhu
(nil)
 
'- flushall to remove all the key that is set by use
127.0.0.1:6379> flushall
OK
127.0.0.1:6379>
 
 
'- Expire the key
 
127.0.0.1:6379> set siddhtime "hello time"
OK
127.0.0.1:6379> get siddhtime
"hello time"
127.0.0.1:6379> expire siddhtime 10
(integer) 1
127.0.0.1:6379> ttl
(error) ERR wrong number of arguments for 'ttl' command
127.0.0.1:6379> ttl siddhtime
(integer) 2
127.0.0.1:6379> ttl siddhtime
(integer) -2
127.0.0.1:6379> get siddhtime
(nil)
127.0.0.1:6379>
 
'- Setting expriration and value for the key in one go
 
127.0.0.1:6379> setex siddhutime 15 "hello"
OK
127.0.0.1:6379> get siddhutime
"hello"
127.0.0.1:6379> ttl siddhutime
(integer) 8
127.0.0.1:6379> get siddhutime
(nil)
127.0.0.1:6379>
 
'- persist to overcome expiration i.e. it will close the expiration clock of the key.
 
127.0.0.1:6379> SETEX siddhutime 180 "siddhu value"
OK
127.0.0.1:6379> persist siddhutime
(integer) 1
127.0.0.1:6379> get siddhutime
"siddhu value"
127.0.0.1:6379>
 
'- Multiple setting values using MSET
 
127.0.0.1:6379> mset siddhu1 "siddhu1 value" siddhu2 "siddhu2 value"
OK
127.0.0.1:6379> get siddhu1
"siddhu1 value"
127.0.0.1:6379> get siddhu2
"siddhu2 value"
127.0.0.1:6379>
 
 
'- Appending the value in key
127.0.0.1:6379> append siddhu1 "appending values"
(integer) 29
127.0.0.1:6379> get siddhu1
"siddhu1 valueappending values"
127.0.0.1:6379>
 
'- Rename the key.
 
127.0.0.1:6379> rename siddhu1 siddhunew1
OK
127.0.0.1:6379> get siddhu1
(nil)
127.0.0.1:6379> get siddhunew1
"siddhu1 valueappending values"
127.0.0.1:6379>
 
 
'- List datatype use lpush to add the value in the list and lrange  to fetch the value from it.
 
127.0.0.1:6379> lpush siddhulist element1
(integer) 1
127.0.0.1:6379> lpush siddhulist element2
(integer) 2
 
 
127.0.0.1:6379> lrange siddhulist 0 10
1) "element2"
2) "element1"
 
To see all the values use below command
127.0.0.1:6379> lrange siddhulist 0 -1
1) "element2"
2) "element1"
 
'- length of the list
 
127.0.0.1:6379> llen siddhulist
(integer) 2
 
'- Remove the item from the list from the top use 
 
127.0.0.1:6379> RPUSH mylist "one" "two" "three" "four" "five"
(integer) 5
127.0.0.1:6379> lpop mylist 1
1) "one"
127.0.0.1:6379> lrange mylist 0 -1
1) "two"
2) "three"
3) "four"
4) "five"
127.0.0.1:6379> lpop mylist 1
1) "two"
127.0.0.1:6379> lrange mylist 0 -1
1) "three"
2) "four"
3) "five"
127.0.0.1:6379> lpop mylist 2
1) "three"
2) "four"
127.0.0.1:6379> lrange mylist 0 -1
1) "five"
 
Same way to remove from the bottom use RPOP
 
 
127.0.0.1:6379> rpop mylist
"five"
127.0.0.1:6379> rpop mylist
"four"
127.0.0.1:6379> lrange mylist 0 -1
1) "one"
2) "two"
3) "three"
127.0.0.1:6379>
 
'- Instert the value in between the key
 
127.0.0.1:6379> linsert mylist before two "newvalue"
(integer) 4
127.0.0.1:6379> lrange mylist 0 -1
1) "one"
2) "newvalue"
3) "two"
4) "three"
127.0.0.1:6379>
 
 
'- SET data type. Set do not allow the duplicate values. to add use sadd and to verify set has the key use sismember.
 
 
127.0.0.1:6379> sadd siddhuset siddhu1
(integer) 1
127.0.0.1:6379> sadd siddhuset siddhu2
(integer) 1
127.0.0.1:6379> sadd siddhuset siddhu3
(integer) 1
127.0.0.1:6379> sismember siddhuset 1
(integer) 0
127.0.0.1:6379> sismember siddhuset siddhu1
(integer) 1
127.0.0.1:6379> sismember siddhuset siddhu2
(integer) 1
127.0.0.1:6379> sismember siddhuset siddhu4
(integer) 0
127.0.0.1:6379>
 
'- to get all the memeber of the set use smembers commands.
 
127.0.0.1:6379> smembers siddhuset
1) "siddhu3"
2) "siddhu1"
3) "siddhu2"
127.0.0.1:6379>
 
'- To get the number of element from the set use below scard command
 
 
127.0.0.1:6379> scard siddhuset
(integer) 3
127.0.0.1:6379> smembers siddhuset
1) "siddhu3"
2) "siddhu1"
3) "siddhu2"
127.0.0.1:6379>
 
'- to find the diff between the two set use sdiff
 
127.0.0.1:6379> smembers siddhuset
1) "siddhu3"
2) "siddhu1"
3) "siddhu2"
127.0.0.1:6379> sadd siddhusetdiff siddhu1
(integer) 1
127.0.0.1:6379> sadd siddhusetdiff siddhu2
(integer) 1
127.0.0.1:6379> sadd siddhusetdiff siddhu4
(integer) 1
127.0.0.1:6379> smembers siddhuset
1) "siddhu3"
2) "siddhu1"
3) "siddhu2"
127.0.0.1:6379> sdiff siddhusetdiff siddhuset
1) "siddhu4"
 
'- To move the value to the other set use smove
 
127.0.0.1:6379> smove siddhuset siddhusetdiff siddhu3
(integer) 1
127.0.0.1:6379> smembers siddhusetdiff
1) "siddhu3"
2) "siddhu1"
3) "siddhu4"
4) "siddhu2"
127.0.0.1:6379>
 
'- To remove element use srem
 
127.0.0.1:6379> srem siddhusetdiff siddhu3
(integer) 1
127.0.0.1:6379> smembers siddhusetdiff
1) "siddhu1"
2) "siddhu4"
3) "siddhu2"
 
'- Sorted set is similar to set  but it have score that we assign to the value. And this score will be used for sorting them in increment or decrement.
 
127.0.0.1:6379> zadd myzset 1 "one" 2 "two" 2 "TWO" 3 "three" 4 "four"
(integer) 5
127.0.0.1:6379> zrange myzset 0 -1
1) "one"
2) "TWO"
3) "two"
4) "three"
5) "four"
 
'- we can also give the decimal value and string for sorting.
 
127.0.0.1:6379> zadd myzset 1.5 "one half" 2.5 "twohalf"
(integer) 2
127.0.0.1:6379> zrange myzset 0 -1
1) "one"
2) "one half"
3) "TWO"
4) "two"
5) "twohalf"
6) "three"
7) "four"
127.0.0.1:6379> zadd myzset 1.3 "one three" 2.3 "two three"
(integer) 2
127.0.0.1:6379> zrange myzset 0 -1
1) "one"
2) "one three"
3) "one half"
4) "TWO"
5) "two"
6) "two three"
7) "twohalf"
8) "three"
9) "four"
 
127.0.0.1:6379> zadd myzset "1.02" "one zero two"
(integer) 1
127.0.0.1:6379> zadd myzset "1.01" "one zero one"
(integer) 1
127.0.0.1:6379> zadd myzset "1.02" "one zero two"
(integer) 0
127.0.0.1:6379> zrange myzset 0 -1
 1) "one"
 2) "one zero one"
 3) "one zero two"
 4) "one two"
 5) "one three"
 6) "one half"
 7) "TWO"
 8) "two"
 9) "two three"
10) "twohalf"
11) "three"
12) "four"
 
'- to find the rank in sorted set use below command
127.0.0.1:6379> zrank myzset one
(integer) 0
127.0.0.1:6379> zrank myzset "one zero one"
(integer) 1
127.0.0.1:6379>
 
'- to increment rank of score use below ZINCRBY command.
 
127.0.0.1:6379> zrange myzset 0 -1
 1) "one"
 2) "one zero one"
 3) "one zero two"
 4) "one two"
 5) "one three"
 6) "one half"
 7) "TWO"
 8) "two"
 9) "two three"
10) "twohalf"
11) "three"
12) "four"
127.0.0.1:6379> ZINCRBY myzset 2 "one"
"3"
127.0.0.1:6379> zrange myzset 0 -1
 1) "one zero one"
 2) "one zero two"
 3) "one two"
 4) "one three"
 5) "one half"
 6) "TWO"
 7) "two"
 8) "two three"
 9) "twohalf"
10) "one"
11) "three"
12) "four"
 
 
'- Hashset to be used to set the key value in string and get the same as shown below.
 
127.0.0.1:6379> HSET siddhuhash name siddhu
(integer) 1
127.0.0.1:6379> HSET siddhuhash email siddhu@gmail.com
(integer) 1
127.0.0.1:6379> hget siddhuhash name
"siddhu"
127.0.0.1:6379> hget siddhuhash email
"siddhu@gmail.com"
127.0.0.1:6379>
 
 
'- to get all from hashset use below command
 
127.0.0.1:6379> hgetall siddhuhash
1) "name"
2) "siddhu"
3) "email"
4) "siddhu@gmail.com"
 
'- for length use hlen
127.0.0.1:6379> hlen siddhuhash
(integer) 2
 
'- to get only key from the hashset use hkeys
 
127.0.0.1:6379> hkeys siddhuhash
1) "name"
2) "email"
127.0.0.1:6379>
 
 
 
'- to get only values from the hashset use hvals
 
127.0.0.1:6379> hvals siddhuhash
1) "siddhu"
2) "siddhu@gmail.com"
127.0.0.1:6379>
 
 
'- to set multiple value in hashset in one command 
 
127.0.0.1:6379> hmset siddhunewhash name "siddhu dhumale" age "43" country "india"
OK
127.0.0.1:6379> hgetall siddhunewhash
1) "name"
2) "siddhu dhumale"
3) "age"
4) "43"
5) "country"
6) "india"
127.0.0.1:6379>

These are the data types supported by Redis

Bitmaps
Cluster
Connection
Geo
Hashes
HyperLogLog
Keys
Lists
Pub/Sub
Scripting
Sentinel
Server
Sets
Sorted Sets
Streams
Strings
Transactions

LHS-BS-HTC-GPSC
list, hash, string, Bitmap, set, sortedset, hyperloglog, transection, connection , Geo, pub/sub, server, streams, sentinel, scripting, Cluster

Here is the java maven project setup for working with Redis using lettuce-core jar.

https://lettuce.io/

Pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>siddhu.redis</groupId>
    <artifactId>siddhu-redis</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>SiddhuRedis</name>
    <description>This is simple Redis example</description>
    <dependencies>
        <dependency>
            <groupId>io.lettuce</groupId>
            <artifactId>lettuce-core</artifactId>
            <version>6.1.6.RELEASE</version>
        </dependency>
    </dependencies>   
</project>

SiddhuRedis.java

/**
 * 
 */
package com.siddhu;
 
import io.lettuce.core.RedisClient;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.sync.RedisCommands;
 
/**
 * @author Siddhartha
 *
 */
public class SiddhuRedis {
 
    /**
     * 
     */
    public SiddhuRedis() {
        // TODO Auto-generated constructor stub
    }
 
    /**
     * @param args
     */
    public static void main(String[] args) {
        // TODO Auto-generated method stub
        RedisClient redisClient = RedisClient.create("redis://password@localhost:6379/0");
        StatefulRedisConnection<String, String> connection = redisClient.connect();
        RedisCommands<String, String> syncCommands = connection.sync();
 
        syncCommands.set("key", "Hello, Redis!");
        syncCommands.hset("siddhuhash1", "siddhuhash2", "siddhuhash3");
         
        System.out.println("syncCommands.hget:"+syncCommands.hget("siddhuhash1", "siddhuhash2"));
        System.out.println("key:"+syncCommands.keys("*"));
        connection.close();
        redisClient.shutdown();
 
    }
 
}

Java World Knowledge Sharing Portal

Monday, January 17, 2022

Redis NoSQL schemaless Mutlidata type supporting Scalable, durable , recoverable Database

No comments:

Total Pageviews

Blog Archive

Contributors