(Translated by https://www.hiragana.jp/)
GitHub - vipshop/pallas: Curator is to Zookeeper what Pallas is to Elasticsearch
Skip to content
/ pallas Public

Curator is to Zookeeper what Pallas is to Elasticsearch

Notifications You must be signed in to change notification settings

vipshop/pallas

Repository files navigation

logo Pallas

Curator is to Zookeeper what Pallas is to Elasticsearch.

                  --Dylan.Xue

Pallas, the VIP.com's uniform search platform, built on top of the elasticsearch, aiming to solve all kinds of search-problem, such as full-text search with scoring, database sharding search, big-data search, complex aggregation search along with performance, scalability and high avaliability required, which are hard to sovled by the tradition SQL or NoSQL database. Shinning features include:

  • ES visualization management & monitoring tools based on Cerebro, bigdesk, sense. You can get them all by visiting http://localhost:8081.
  • ES search template management based on index level.
    • CURD, render and debug the templates.
    • Import and export.
    • Include syntax, highlighting, json fromat.
    • Review process required before taking effect.
    • Version comparing.
    • Template canary.
  • Deep integration between MySQL and ES.
    • Field Mapping auto generated based on MySQL schema.
    • Generate DSL via elasticsearch-sql.
    • Data-sync from DB to ES, including full-import/checking/binlog-based incremental update.(base on saturn and rdp, to be open)
  • ES http-protocol proxy.
    • Netty-based application with filter-chain pattern.
    • Service governance: index routing/timeout/retry/flow-record/slow log/index and template canary/toke-based authorization.
    • Together with Client-SDK provides you with load balance/failover abilities.
  • Pallas ES-plugin updated without a restarat.
    • Extends from AbstractSearchScript.
    • Each plugin has its own classloader.
    • Thorough class loading & unloading.
  • Pallas Console
    • ES clusters management.
    • Index management.
    • Tempalate review process.
    • ES-proxy management(metrisc/offline & online).
    • Token management.
    • Pallas plugin management.

Pallasただひんかい统一てき检索平台ひらだい于ElasticSearch深度しんどじょうせいかい决关けいすうすえ库和NOSQLひとし无法有效ゆうこう处理てきぶんほん检索ぶん模糊もこ查询,すうすえ关键组合查询,聚合とう问题场景。 こうのう包括ほうかつ:

  • ES视化管理かんり监控,Cerebrobigdesksenseつう过访问http://localhost:8081 就能ぜん拥有りょう
  • ESばん管理かんり索引さくいんつぶたび
    • ぞう删改查、渲染DSL、ざい线调试
    • 批量导入导出
    • include语法, json 美化びかこうあきら语法
    • ばん审核つくえせいはいじょう线
    • 新旧しんきゅう版本はんぽん对比
  • 深度しんど整合せいごうMySQLES
    • もと于db schema生成せいせいField mapping
    • つうelasticsearch-sql动转SQLいたES DSL。
    • かずすえどう全量ぜんりょう、对账、增量ぞうりょう),saturn rdp,这块很快かい开源。
  • ES http协议代理だいり
    • もと于nettyfilter链模しき代理だいりhttp请求
    • かんぜんてきふく务治みちゆかりちょう时重试、流量りゅうりょう录制、索引さくいんばんはいもと于tokenてき鉴权
    • 配合はいごう使用しようclient-sdk,のう做到负载均衡きんこう动容错
  • PallasてきES插件,线上热插拔
    • Extends from AbstractSearchScript
    • 独立どくりつてきclassloader设计
    • かんぜんてきclass载和おろし
  • Pallas Console管理かんりひかえせいだい
    • ESしゅうぐん管理かんり
    • 代理だいり管理かんり(监控ゆび标、うえ线下线)
    • token管理かんり
    • ばん审核りゅうほど
    • 插件管理かんり

Main modules 主要しゅよう

Modules Functionalities
pallas-es もと于ES 5.5.2
增加ぞうかざい线升级、ざい线重启插けん
提供ていきょうrpmつつめいち键安そう
pallas-index
なお开源
どう步数ほすうすえ块,包括ほうかつ全量ぜんりょう增量ぞうりょう、对账
どうじょう态检测
pallas-search ES http代理だいり
ふく务治:鉴权、みちゆかりちょう时重试、流量りゅうりょう复制、ばんはい、别名はいとう
pallas-console 索引さくいん管理かんり管理かんり索引さくいん管理かんりばん管理かんり版本はんぽん流量りゅうりょう录制、みちよし管理かんりふく务治索引さくいん操作そうさ记录とうとう
しゅうぐん管理かんり管理かんりESあつまりぐん集成しゅうせいcerebro视化管理かんりけい统,集成しゅうせいbigdesk视化监控
权限管理かんりもと于tokenてき授权方式ほうしき
代理だいり管理かんりうえ线/しも线代理だいり,实时查看qps/流量りゅうりょう记录/连接すう
pallas-client もと于ES rest client
动负载均衡きんこう
ちょう时重试
配置はいちちょく连ES还是经过proxy

Pallas architecture

arc

How it looks

looks

looks

looks

looks

Quick Start

Visit http://localhost:8081 after running the following command, make sure you have >=jdk8 installed.

cd quickstart
java -Dfile.encoding=UTF-8 -jar pallas-demo.jar  
(あるもの运行:sh quickstart.sh)

Click here for more details: Quick Start

Contributing

Pallasかんかたほろしん讨论ぐん,请搜索そうさくほろ信号しんごうviptech128(ただわざ术),添加てんかこうとも(备注Pallas开源交流こうりゅうきさき加入かにゅう

所有しょゆう报Bug、けん议与咨询,请在Issue发起;所有しょゆうだい码提交,请走Pull Requestりゅうほど

对于优秀てきだい码提交和けん议,ただひんかいはたしわ发挥电商ほんしょく,给予vip.com购物卡的奖励 !!!

Documents

Get Started

Team members