Apache Doris 百度自研的交互式 SQL 數據倉庫--集群安裝

Apache Doris 百度自研的交互式 SQL 數據倉庫--集群安裝

Doris社區及用戶

安裝

安裝jdk

安裝c

互通網絡

關閉防火牆

一.配置

FE: 192.168.28.131 10G(內存) 100G

BE1: 192.168.28.128 20G(內存) 200G

BE2: 192.168.28.129 20G(內存) 200G

BE3: 192.168.28.130 20G(內存) 200G

Ⅰ).BE

vi be.conf

# INFO,WARNING,ERROR,FATAL

sys_log_level=INFO

# ports for admin,web,heartbeat service

be_port=9060

be_rpc_port=9070

webserver_port=8040

heartbeat_service_port=9050

brpc_port=8060

# Choose one if there are more than one ip except loopback address

# 24為網關

priority_networks=hostname/24 (hostname修改對應be的ip)

# storage_root_path=/home/disk1/palo.HDD,50;/home/disk2/palo.SSD,1;/home/disk2/palo

# 目錄需手動創建

storage_root_path=/data/opt/be/palo


Ⅱ).FE

vi fe.conf

# INFO,WARNING,ERROR,FATAL

sys_log_level=INFO

# store metadata,create it if it is not exist

meta_dir=${DORIS_HOME}/palo-meta

http_port=8030

rpc_port=9020

query_port=9030

edit_log_port=9010

# Choose one if there are more than one ip except loopback address

# 24為網關

priority_networks=hostname/24(hostname修改對應fe的ip)


Ⅲ).網絡通信端口


Apache Doris 百度自研的交互式 SQL 數據倉庫--集群安裝

二.服務命令

Ⅰ).啟停

a).啟動服務FE和BE

sh /data/opt/fe/bin/start_fe.sh --daemon

sh /data/opt/be/bin/start_be.sh --daemon

b).停止服務

sh /data/opt/fe/bin/stop_fe.sh

sh /data/opt/be/bin/stop_be.sh

Ⅱ).驗證

URL: http://hostname:8030/system 查看FE節點: http://hostname:8030/system?path=//frontends


三.業務命令

Ⅰ).連接FE

mysql -h 192.168.0.11 -P 9030 -uroot -p123456

Ⅱ).創建數據庫

create database doris;

Ⅲ).創建用戶

create user 'doris' identified by '1234';

Ⅳ).賦權

grant all on doris to doris;

Ⅴ).添加BE

ALTER SYSTEM ADD BACKEND "hostname:9050";

Ⅵ).查看BE狀態

SHOW PROC '/backends';


使用 mysql-client 連接到 FE,並執行

SHOW PROC '/backends';

查看 BE 運行情況。如一切正常,isAlive 列應為 true

Ⅶ).刪除BE

ALTER SYSTEM DROPP BACKEND "hostname:9050";

Ⅷ).添加FE

ALTER SYSTEM ADD FOLLOWER "hostname:9050";

Ⅸ).查看FE狀態

SHOW PROC '/frontends';

Ⅹ).刪除FE

ALTER SYSTEM DROPP FOLLOWER "hostname:9050";

Ⅺ).創建表

CREATE TABLE doris.table2

(

event_day DATE,

siteid INT DEFAULT '10',

citycode SMALLINT,

username VARCHAR(32) DEFAULT '',

pv BIGINT SUM DEFAULT '0'

)

AGGREGATE KEY(event_day,citycode,username)

PARTITION BY RANGE(event_day)

(

PARTITION p1 VALUES LESS THAN ('2019-06-30'),

PARTITION p1 VALUES LESS THAN ('2019-07-31'),

PARTITION p1 VALUES LESS THAN ('2019-08-31')

)

DISTRIBUTED BY HASH(siteid) BUCKETS 10

PROPERTIES("replication_num"="1")


Ⅻ).導入數據

1.database: doris 2.tablename: table2 3.label: 每個批次不可重複 4.column_separator: 數據文件中字段之間特殊符號分割(|-%7c, ,-%2c, t-%09, x01-%01)

分割符

對應值

豎線

%7c

,

%2c

\\t

%09

\\\\x01

%01

curl --location-trusted -u db_user -T path/filename.txt http://hostname/8030/api/doris/table2/_load?label=100\\&column_separator=%7c


curl --location-trusted -u root:root -T aaa http://192.168.1.5:8030/api/example_db/NewTable2/_load?label=table2_20190901\\&column_separator=%2c -V


XIII).創建MySQL引擎表

CREATE TABLE IF NOT EXISTES doris.table_name

(

'agent_id' INT NOT NULL,

'accept_time' DATETIME NOT NULL,

'signature' VARCHAR(770) NOT NULL,

'bad_app_count' BIGINT,

'total_count' BIGINT,

'version' INT DEFAULT "0",

'method_type' VARCHAR(20)

)

ENGINE=mysql

PROPERTIES

(

"host"="db_hostname",

"port"="3306",

"user"="db_user",

"password"="password",

"database"="database",

"table"="table_name"

)

Apache Doris 百度自研的交互式 SQL 數據倉庫--集群安裝


分享到:


相關文章: