python-client

pegasus-python-client

It uses Twisted for the asynchronous communication with pegasus server.

Installation

Python 3.8+

pypegasus can be installed via pip as follows:

git clone https://github.com/apache/incubator-pegasus
cd inclubator-pegasus/python-client
thrift --gen py -out . ../idl/rrdb.thrift
thrift --gen py -out . ../idl/dsn.layer2.thrift
python3 setup.py install

Usage

There are some basic guide in sample.py.

Test

Note: Before testing, you should firstly start an onebox.

Run sample

python3 sample.py

Basic interfaces test

cd tests
python3 -m unittest test_basics.TestBasics

Integration test

Note: You should firstly set proper Pegasus shell path in test_integration.py.

shell_path = '/your/pegasus-shell/dir'

cd tests
python3 -m twisted.trial test_integration.py

Benchmark test

cd tests
python3 -m unittest test_benchmark.TestBasics

The test result on my personal PC (CPU: Intel i7-7700 3.60GHz, mem: 8G) is:

10000 get cost: 3.80077195168 s, 0.000380077195168 s per op
10000 remove cost: 7.61887693405 s, 0.000761887693405 s per op
10000 set cost: 6.26366090775 s, 0.000626366090775 s per op

FAQ

Q: Error occurred about 'module' object has no attribute 'OP_NO_TLSv1_1':

Error details:

Traceback (most recent call last):
  File "./sample.py", line 3, in <module>
    from pgclient import *
  File "/home/lai/dev/pegasus-python-client/pgclient.py", line 7, in <module>
    from twisted.internet import reactor
  ...
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/_sslverify.py", line 38, in <module>
    TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1,
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'

A: pyOpenSSL version too low

The twisted version we used is 17.9.0, which requires pyOpenSSL>=16.0.0, you can

pip install --upgrade pyopenssl

ref: scrapy/scrapy#2473