搜索(search)模块JAVA版.html

Elasticsearch是一个基于Lucene构建的开源、实时、分布式、RESTful接口的全文搜索引擎和分析引擎。Elasticsearch还是一个分布式文档数据库，其中每个字段均可被索引，而且每个字段的数据均可被搜索，Elasticsearch能够横向扩展至数以百计的服务器存储以及处理PB级的数据。可以在极短的时间内存储、搜索和分析大量的数据。它是一个提供了分布式多用户能力的全文搜索引擎，通常作为具有复杂搜索场景情况下的核心发动机，可以用于全文检索、结构化检索和分析，并能将这三者结合起来，是现在使用最广的开源搜索引擎。

新建一个maven项目

导入配置文件

<dependencies>
<!--thrift相关-->
<dependency>
    <groupId>org.apache.thrift</groupId>
    <artifactId>libthrift</artifactId>
    <version>0.11.0</version>
</dependency>
<!--junit-->
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
</dependency>
<!--fastjson-->
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.2.28</version>
</dependency>
    </dependencies>

导入thrift相关接口

com.dksou.fitting.search.SearchService.SearchService.java com.dksou.fitting.search.SearchIOService.FileData.java com.dksou.fitting.search.SearchIOService.DKSearchOutput.java com.dksou.fitting.search.SearchIOService.DKSearchInput.java

模块配置：properties配置文件

#thrift服务端开发端口
dkSearch.port=9875

#commons-pool2连接池配置
#默认配置maxTotal是8，池中有8个client
dkSearch.commons-pool.MaxTotal=10
#默认配置最大空闲数
dkSearch.commons-pool.MaxIdlel=10
#默认配置最小空闲数
dkSearch.commons-pool.MinIdle=0
#最大等待时间
dkSearch.commons-pool.MaxWaitMillis=5000

#是否开启自动探测集群
client.transport.sniff=true
#searchguard配置，如果没有就不用写
#是否启动ssl加密连接
#searchguard.ssl.transport.enabled=true
#searchguard秘钥库路径
#searchguard.ssl.transport.keystore_filepath=/opt/fittingsearch/conf/admin-keystore.jks
#searchguard秘钥库密码
#searchguard.ssl.transport.keystore_password=123456
#searchguard信任库路径
#searchguard.ssl.transport.truststore_filepath=/opt/fittingsearch/conf/truststore.jks
#searchguard信任库密码
#searchguard.ssl.transport.truststore_password=123456
#是否开启主机名验证
#searchguard.ssl.transport.enforce_hostname_verification=false
#是否忽略集群名称验证
#client.transport.ignore_cluster_name=true
searchguard.ssl.transport.enabled=
searchguard.ssl.transport.keystore_filepath=
searchguard.ssl.transport.truststore_password=
searchguard.ssl.transport.truststore_filepath=
searchguard.ssl.transport.keystore_password=
searchguard.ssl.transport.enforce_hostname_verification=
client.transport.ignore_cluster_name=

ES导入导出

工具类名：DKSearchInput 该方法可以实现过大文件导入指定大小部分，只截取指定大小的文件内容导入到搜索引擎当中。文件型数据（txt、doc、xls、pdf要解析其里面的内容）导入ES

本地文件导入（fitting-search 组件所在服务器）
- 方法签名：String file2ES(int fileType, String filePath, String hostIps, String clusterName, String indexName, String typeName, int port, int length);
- 返回：成功返回SUCCESS字符串，错误返回异常
- 签名参数说明： fileType:文件类型（1-txt，2-doc，3-xls，4-pdf）， filePath为服务端本地文件所在的目录（可嵌套子目录，目录名称不能为中文）， hostIps：要连接搜索主机的ip地址列表， clusterName为搜索引擎集群名称， indexName搜索引擎的索引名称（自定义），索引存在话会往已有索引里添加，如果没有的话会创建， typeName搜索引擎的类型名称（自定义），port搜索引擎的端口号，fileLength文件长度限制（K为单位。）。
- 范例：把本地文件夹" E:\fitting\test数据 "下，指定类型的文件导入到索引名为"file2es"、类型名为"fileType"的搜索引擎中，可以使用file2ES方法实现。
- 代码案例：

import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
/**
 * @create: 10-22-16
 * @author:
 **/
public class ClientInput{
    TTransport tTransport;
    DKSearchInput.Client clientInput = null;
    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.1.5", 9875);
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchInput = new TMultiplexedProtocol( tProtocol, "DKSearchInput" );
        clientInput = new DKSearchInput.Client( DKSearchInput );
        tTransport.open();
    }
    @After
    public void close() {
        //关闭连接
        tTransport.close();
    }
    @Test
    public void  ClientInput() {
        int fileType =4;
        String dirname="E:\\fitting\\test数据\\pdf测试\\自控力";
        String hostIps="192.168.43.109,192.168.43.110,192.168.43.111";
        int port=9300;
        String clustername="node09";
        String indexName="test3";
        String typeName="txt";
        int length=10000;
        try {
            String nosql2ES= clientInput.file2ES( fileType,dirname,hostIps,clustername,indexName,typeName,port,length );
            if (nosql2ES .equals( "SUCCESS")) {
                System.err.println( "数据导入成功！ SUCCESS!"+nosql2ES );
            }else {
                System.err.println( "数据导入失败！ failed!"+nosql2ES );
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

客户端本地文件导入

本地文件导入（fitting-search 组件所在服务器）
- 方法签名：String file2ESByBinary(String fileName, String fileDir, String hostIps, String clusterName, String indexName, String typeName, int port, FileData fileData, int fileType)
- 返回：成功返回SUCCESS字符串，错误返回异常
- 签名参数说明：
  - @param fileName 文件名字（尽量不使用中文）
  - @param fileDir 文件在服务端暂存的目录，目录名称不能是中文
  - @param hostIps ES集群的IP
  - @param clusterName ES集群的名称
  - @param indexName ES的索引名称
  - @param typeName ES的类型名称
  - @param port 端口号
  - @param fileData 文件数据包括（文件名称，二进制数据）
  - @param fileType 文件类型，1-txt，2-doc，docx，3-xls，xlsx，4-pdf
  - @param filePath 客户端本地文件的地址
- 范例：把本地文件夹" E:\fitting\test数据 "下，指定类型的文件导入到索引名为"file2es"、类型名为"fileType"的搜索引擎中，可以使用file2ES方法实现。
- 代码案例：

import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.nio.ByteBuffer;
/**
 * @create: 10-26-17
 * @author:
 **/
public class asd {
    private TTransport tTransport;
    DKSearchInput.Client clientInput = null;
    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.43.111", 9875);
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchInput = new TMultiplexedProtocol( tProtocol, "DKSearchInput" );
        clientInput = new DKSearchInput.Client( DKSearchInput );
        tTransport.open();
    }
    @After
    public void close() {
        //关闭连接
        tTransport.close();
    }
    @Test
    public void  FileESBybinary() {
        String fileName ="influence.pdf";
        String dirname="/home/";
        String hostIps="192.168.43.111";
        int port=9301;
        String clustername="node";
        String indexName="test1";
        String typeName="pdfff";
        String filePath = "E:\\fitting\\test数据\\pdf测试\\[影响力].(influence).罗伯特·西奥迪尼.文字版.pdf";
        try {
            // 构造文件数据
            byte[] bytes = fileToBytes(filePath);
            FileData fileData = new FileData();
            fileData.name = filePath;
            fileData.buff = ByteBuffer.wrap(bytes);
            String nosql2ES=clientInput.file2ESByBinary
                    ( fileName,dirname,hostIps,clustername,indexName,typeName,port,fileData,4 );
            if (nosql2ES .equals( "SUCCESS")) {
                System.err.println( "数据导入成功！ SUCCESS!"+nosql2ES );
            }else {
                System.err.println( "数据导入失败！ failed!"+nosql2ES );
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    private static byte[] fileToBytes(String filePath) {
        byte[] buffer = null;
        try {
            File file = new File(filePath);
            FileInputStream fis = new FileInputStream(file);
            byte[] b = new byte[(int) file.length()];
            ByteArrayOutputStream bos = new ByteArrayOutputStream(b.length);
            int n;
            while ((n = fis.read(b)) != -1) {
                bos.write(b, 0, n);
            }
            fis.close();
            bos.close();
            buffer = bos.toByteArray();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return buffer;
    }
}

ES文件导出本地（txt文本）
- 方法签名：ES2Txt(String hostIps, String clusterName, String indexName, String typeName, int port, int start, int size)
- 返回：成功返回SUCCESS字符串，错误返回异常
- 签名参数说明：
  - 导出ES数据.txt类型
  - @param hostIps ES的ip地址
  - @param clusterName ES的集群名称
  - @param indexName ES的索引名称
  - @param typeName ES的类型名称
  - @param port ES的端口号
  - @param start 从哪儿开始
  - @param size 到哪儿结束
- 范例：把es中数据导出到本地，可以使用ES2Txt方法实现。
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

/**
 * @create: 10-26-10
 * @author:
 **/
public class ESTXT {
    TTransport tTransport;
    DKSearchOutput.Client clientOutput = null;
    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.1.5", 9875);
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchOutput = new TMultiplexedProtocol( tProtocol, "DKSearchOutput" );
        clientOutput = new DKSearchOutput.Client( DKSearchOutput );
        tTransport.open();
    }

    @After
    public void close() {
//关闭连接
        tTransport.close();
    }
    @Test
    public void ESTXT() throws TException {
        String hostIps="192.168.43.109,192.168.43.110,192.168.43.111";
        int port=9300;
        String clustername="node09";
        String indexName="test3";
        String typeName="txt";
        int form=0;
        int size=2600;
        try {
            String s = clientOutput.ES2Txt( hostIps, clustername, indexName, typeName, port, form, size);
            if(s.equals("SUCCESS")) {
                System.err.println("数据导出成功！ SUCCESS!");
            }else {
                System.err.println("数据导出失败！"+ s);
            }
        }catch (Exception e){
            e.printStackTrace();
        }
    }
}

ES文件导出本地（excel文本）
- 方法签名：String ES2XLS(String hostIps, String clusterName, String indexName, String typeName, int port, int start, int size)
- 返回：成功返回SUCCESS字符串，错误返回异常
- 签名参数说明：
  - 导出ES数据.xlsx类型
  - 导出ES数据 .xls类型
  - @param hostIps ES的ip地址
  - @param clusterName ES的集群名称
  - @param indexName ES的索引名称
  - @param typeName ES的类型名称
  - @param port ES的端口号
  - @param start 从哪儿开始
  - @param size 到哪儿结束
- 范例：把es中数据导出到本地生成excel，可以使用ES2Txt方法实现。
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

/**
 * @create: 10-26-11
 * @author:
 **/
public class ES2XLS {
    TTransport tTransport;
    DKSearchOutput.Client clientOutput = null;
    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.1.5", 9875);
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchOutput = new TMultiplexedProtocol( tProtocol, "DKSearchOutput" );
        clientOutput = new DKSearchOutput.Client( DKSearchOutput );
        tTransport.open();
    }
    @After
    public void close() {
//关闭连接
        tTransport.close();
    }
    @Test
    public void  excel(){
        String hostIps="192.168.43.109,192.168.43.110,192.168.43.111";
        int port=9300;
        String clustername="node09";
        String indexName="test3";
        String typeName="txt";
        int form=0;
        int size=1000;
        try {
            String es2XLS = clientOutput.ES2XLS( hostIps, clustername, indexName, typeName, port, form, size );
            if (es2XLS.equals( "SUCCESS" )) {
                System.out.println( "导出成功"+es2XLS );
            }else {
                System.out.println("导出失败"+es2XLS);
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }
}

精准查询
- 方法签名：Map<String, String> ConditionalQuery(String hostIps, String clusterName, String indexName, String typeName, int port, int start, int size, String sentence)
- 返回：map集合，包括命中数和命中的数据
- 签名参数说明：
  - 条件查询，达到1500提前结束，设置是否按查询匹配度排序，精确查询
  - @param hostIps ES集群的ip地址
  - @param clusterName ES集群集群名称
  - @param indexName ES集群的索引名称，可使用多个索引 indexName="test2,test1,test";
  - @param typeName 索引类型，可多个 typeName="doc,pdf,test";
  - @param port ES集群的端口号
  - @param start 记录偏移 , null-默认为0
  - @param size 记录偏移 , null-默认为10
  - @param sentence 需要搜索的词语，
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

import java.util.Map;

/**
 * @create: 10-25-10
 * @author:
 **/
public class ConditionalQuery {
    TTransport tTransport;
    SearchService.Client client = null;

    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.1.160", 9875 );
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchService = new TMultiplexedProtocol( tProtocol, "DKSearchService" );
        client = new SearchService.Client( DKSearchService );
        tTransport.open();
    }
    @After
    public void close() {
        tTransport.close();
    }
    @Test
    public void SearchService() {
        String hostIps = "192.168.1.2";
        int port = 9300;
        String clustername = "dkes";
        String indexName = "test3";
        String typeName = "pdf";
        int start = 0;
        int size = 10;
        String sentence = "张飞";

        try {
            Map<String, String> searchcahange = client.ConditionalQuery( hostIps, clustername, indexName, typeName, port, start, size, sentence );
            for (Map.Entry map :
                    searchcahange.entrySet()) {
                System.out.println( map.getKey() + "======" + map.getValue() );
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }

}

过滤查询
- 方法签名：Map<String, String> FileterQuery(String hostIps, String clusterName, String indexName, String typeName, int port, int start, int size, String field, String gte, String lte)
- 返回：map集合，包括命中数和命中的数据
- 签名参数说明：
  - 过滤查询，可添加多条过滤条件
  - @param hostIps ES集群的ip地址
  - @param clusterName ES集群集群名称
  - @param indexName ES集群的索引名称，可使用多个索indexName="test2,test1,test";
  - @param typeName 索引类型，可多个 typeName="doc,pdf,test";
  - @param port ES集群的端口号
  - @param start 记录偏移 , null-默认为0
  - @param size 记录偏移 , null-默认为10
  - @param field 字段名，例如："age,balance"
  - @param gte 过滤条件(大于等于) 例如："20,1000"
  - @param lte 过滤条件(小于等于) 例如："23,2000"
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

import java.util.Map;

/**
 * @create: 10-25-11
 * @author:
 **/
public class FileterQuery {
    TTransport tTransport;
    SearchService.Client client = null;

    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.1.160", 9875 );
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchService = new TMultiplexedProtocol( tProtocol, "DKSearchService" );
        client = new SearchService.Client( DKSearchService );
        tTransport.open();
    }
    @After
    public void close() {
        tTransport.close();
    }

    @Test
    public void testFileterQuery() {
        String hostIps = "192.168.1.5";
        int port = 9300;
        String clustername = "node";
        String indexName = "bank";
        String typeName = "account";
        int start = 2;
        int size = 10;
        String field = "age";
        String gte="20";
        String lte="23";
        try {
            Map<String, String> aggregate = client.FileterQuery( hostIps,  clustername,  indexName,  typeName,  port,  start,  size,  field,  gte,lte);

            for (Map.Entry map :
                    aggregate.entrySet()) {
                System.out.println( map.getKey() + "======" + map.getValue() );
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }

}

统计
- 方法签名：Map<String, String> StatsAggregation(String hostIp, int port, String clusterName, String indexName, String typeName, String aggFdName, String aggType)
- 返回：map集合，包括sum，max，min，avg等。
- 签名参数说明：
  - 聚合统计，
  - @param hostIp ES集群的ip地址
  - @param clusterName ES集群集群名称
  - @param indexName ES集群的索引名称，可使用多个索indexName="test2,test1,test";
  - @param typeName 索引类型，可多个 typeName="doc,pdf,test";
  - @param port ES集群的端口号
  - @param aggFdName 需要统计的字段
  - @param aggType 记录偏移 , null-默认为10
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

import java.util.Map;

/**
 * @create: 10-25-11
 * @author:
 **/
public class StatsAggregation {
    TTransport tTransport;
    SearchService.Client client = null;
    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.43.111", 9875 );
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchService = new TMultiplexedProtocol( tProtocol, "DKSearchService" );
        client = new SearchService.Client( DKSearchService );
        tTransport.open();
    }
    @After
    public void close() {
        tTransport.close();
    }
    @Test
    public void testStatsAggregation() {
        String hostIps = "node09,node10,node11";
        int port = 9300;
        String clustername = "node";
        String indexName = "bank";
        String typeName = "account";
        String aggFdName = "balance";
        String aggType = "null";
        try {
            Map<String, String> aggregate = client.StatsAggregation( hostIps, port, clustername, indexName, typeName, aggFdName, aggType);
            for (Map.Entry map :aggregate.entrySet()) {
                System.out.println( map.getKey() + "======" + map.getValue() );
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }

}

百分比查询
- 方法签名：Map<String, String> PercentilesAggregation(String hostIp, int port, String clusterName, String indexName, String typeName, String aggFdName, String percentiles)
- 返回：map集合。
- 签名参数说明：
  - 对指定字段（脚本）的值按从小到大累计每个值对应的文档数的占比（占所有命中文档数的百分比），返回指定占比比例对应的值。默认返回[ 1, 5, 25, 50, 75, 95, 99 ]分位上的值。如下中间的结果，可以理解为：占比为50%的文档的age值 <= 31，或反过来：age<=31的文档数占总命中文档数的50% 也可以自己指定分位值，如 10,50,90.
  - @param hostIp ES集群的ip地址
  - @param port ES集群的端口号
  - @param clusterName ES集群名称
  - @param indexName ES集群的索引名称，可使用多个索indexName="test2,test1,test";
  - @param typeName 索引类型，可多个 typeName="doc,pdf,test";
  - @param aggFdName 字段名称
  - @param percentiles 自定义百分比分位数，比如10,50,90 之间以","分隔
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

import java.util.Map;

/**
 * @create: 10-26-16
 * @author:
 **/
public class PercentilesAggregation {
    TTransport tTransport;
    SearchService.Client client = null;

    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.43.111", 9875 );
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchService = new TMultiplexedProtocol( tProtocol, "DKSearchService" );
        client = new SearchService.Client( DKSearchService );
        tTransport.open();
    }
    @After
    public void close() {
        tTransport.close();
    }

    @Test
    public void testPercentilesAggregation() {
        String hostIps = "node09,node10,node11";
        int port = 9300;
        String clustername = "node09";
        String indexName = "bank";
        String typeName = "account";
        String aggFdName="age";
        String aggType="10,50,90";
        try {
            Map<String, String> aggregate = client.PercentilesAggregation( hostIps, port, clustername,  indexName,  typeName,  aggFdName,aggType);
            for (Map.Entry map :aggregate.entrySet()) {
                System.out.println( map.getKey() + "======" + map.getValue() );
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }

}

根据字段值项分组聚合
- 方法签名：Map<String, String> TermsAggregation(String hostIp, int port, String clusterName, String indexName, String typeName, String aggFdName)
- 返回：map集合。
- 签名参数说明：
  - Terms Aggregation 根据字段值项分组聚合
  - @param hostIp ES集群的IP
  - @param port ES集群的端口号
  - @param clusterName ES集群的名称
  - @param indexName ES的索引名称
  - @param typeName ES的类型名称
  - @param aggFdName 需要聚合的字段
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

import java.util.Map;

/**
 * @create: 10-26-16
 * @author:
 **/
public class TermsAggregation {
    TTransport tTransport;
    SearchService.Client client = null;

    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.43.111", 9875 );
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchService = new TMultiplexedProtocol( tProtocol, "DKSearchService" );
        client = new SearchService.Client( DKSearchService );
        tTransport.open();
    }
    @After
    public void close() {
        tTransport.close();
    }

    @Test
    public void testTermsAggregation() {
        String hostIps = "node09,node10,node11";
        int port = 9300;
        String clustername = "node09";
        String indexName = "bank";
        String typeName = "account";
        String aggFdName="age";
        try {
            Map<String, String> aggregate = client.TermsAggregation( hostIps, port, clustername,  indexName,  typeName,  aggFdName);
            for (Map.Entry map : aggregate.entrySet()) {
                System.out.println( map.getKey() + "======" + map.getValue() );
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }

}

根据多条件组合查询
- 方法签名：Map<String, String> MustSearch(String hostIp, int port, String clusterName, String indexName, String typeName, String filed, String filedName, int start, int size)
- 返回：map集合。
- 签名参数说明：
  - 根据多条件组合与查询（相当于sql里的and,age=32 and gender=m,也可以单个条件查询age=32. ）
  - @param hostIp ES集群的IP地址
  - @param port 连接ES集群的端口号
  - @param clusterName ES集群的名称
  - @param indexName ES集群的索引
  - @param typeName ES集群的类型名称
  - @param filed 查询的字段名称比如：age，gender
  - @param filedName 查询的内容比如：32，m
  - @param start 记录偏移 , null-默认为0
  - @param size 记录偏移 , null-默认为10
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

import java.util.Map;

/**
 * @create: 10-26-17
 * @author:
 **/
public class MustSearch {
    TTransport tTransport;
    SearchService.Client client = null;

    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.43.111", 9875 );
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchService = new TMultiplexedProtocol( tProtocol, "DKSearchService" );
        client = new SearchService.Client( DKSearchService );
        tTransport.open();
    }
    @After
    public void close() {
        tTransport.close();
    }

    @Test
    public void testMustSearch() {
        String hostIps = "node09,node10,node11";
        int port = 9300;
        String clustername = "node09";
        String indexName = "bank";
        String typeName = "account";
        String filed="age,gender";
        String filedName="32,m";
        try {
            Map<String, String> aggregate = client.MustSearch( hostIps,port,clustername,indexName,typeName,filed,filedName,0,10);
            for (Map.Entry map : aggregate.entrySet()) {
                System.out.println( map.getKey() + "======" + map.getValue() );
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }

}

多条件或查询
- 方法签名：Map<String, String> ShouldSearch(String hostIp, int port, String clusterName, String indexName, String typeName, String filed, String filedName, int start, int size)
- 返回：map集合。
- 签名参数说明：
  - 多条件或查询（相当于sql里的or，age=32 or gender=m,也可以单个条件查询age=32.）
  - @param hostIp ES集群的IP
  - @param port ES集群的端口号
  - @param clusterName ES集群名称
  - @param indexName 索引
  - @param typeName 类型
  - @param filed 字段比如：age，gender
  - @param filedName 查询内容比如：32，m
  - @param start 记录偏移
  - @param size 记录偏移
- 代码案例：

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TMultiplexedProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import com.dksou.fitting.search.SearchService.SearchService;
import com.dksou.fitting.search.SearchIOService.FileData;
import com.dksou.fitting.search.SearchIOService.DKSearchOutput;
import com.dksou.fitting.search.SearchIOService.DKSearchInput;

import java.util.Map;

/**
 * @create: 10-26-17
 * @author:
 **/
public class ShouldSearch {
    TTransport tTransport;
    SearchService.Client client = null;

    @Before
    public void Before() throws TTransportException {
        tTransport = new TSocket( "192.168.43.111", 9875 );
        TProtocol tProtocol = new TBinaryProtocol(tTransport);
        TMultiplexedProtocol DKSearchService = new TMultiplexedProtocol( tProtocol, "DKSearchService" );
        client = new SearchService.Client( DKSearchService );
        tTransport.open();
    }
    @After
    public void close() {
        tTransport.close();
    }

    @Test
    public void testShouldSearch() {
        String hostIps = "node09,node10,node11";
        int port = 9300;
        String clustername = "node09";
        String indexName = "bank";
        String typeName = "account";
        String filed="age,gender";
        String filedName="32,m";
        try {
            Map<String, String> aggregate = client.ShouldSearch( hostIps,port,clustername,indexName,typeName,filed,filedName,0,10);
            for (Map.Entry map : aggregate.entrySet()) {
                System.out.println( map.getKey() + "======" + map.getValue() );
            }
        } catch (TException e) {
            e.printStackTrace();
        }
    }

}

官网地址：http://www.dksou.com

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

搜索(search)模块JAVA版.html

Fitting V1.0 文档

目录

用户安装使用手册

一、概述

二、Fitting打包说明

三、安装说明

软硬件环境

软件的安装卸载和升级

四、操作说明

用户管理

Fitting集群组件管理

安装主机状态监控功能

用户开发使用手册JAVA版

一、开发依赖说明JAVA版

二、数据源(datasource)模块JAVA版

三、SQL工具类(sqlutils)模块JAVA版

四、实时流(stream)模块JAVA版

五、搜索(search)模块JAVA版

六、自然语言处理(nlp)模块JAVA版

七、机器学习(ml)模块JAVA版

八、图计算(graphx)模块JAVA版

九、ElasticSQL模块JAVA版

十、数据处理(dataprocess)模块JAVA版

Clone this wiki locally