Darknet Markets Analysis & Business Intelligence

In my honor thesis of darknet markets analysis and business intelligence, I investigated the relation between sellers' lisitngs similarities and sellers' levels using Gensim, a robust open-source vector space modeling and topic modeling toolkit implemented in Python. First, I parsed 13,828 web pages using regular expression to obatin all sample data. Then, I calculated similarities scores of each pair of sellers using dov2vec as a part of Gensim for purposes of topic modelling, document indexing and similarity retrieval with large corpora. By calculating correlations of listing similarities and seller levels of all sellers, the final result of -0.0031 indicates that there is no relation between sellers' listings similarities and sellers' levels.

The full presentation vedio or a short demo

Situation (S):

At W.P.Carey School of Business's Actionable Analytics Lab, professor Victor Benjamin ([email protected]) focused on providing rigorous solutions to real-world problems such as darnket markets. I liked challenge myself on meaningful projects so I connacted Dr. Benjamin for being my thesis director on the project of darknet markets and business intelligence.

Task (T):

My goal was to investigate the relation between sellers' lisitngs similarities and sellers' levels by processing a large scale of data (13,828 web pages) using Deep Learning approach in Python language.

Action (A):

I quickly studied Python myself by watching online tutoirals and asking my director for advice. After successfully attempting gensim on the hansa market (another famous darknet market) with a small dataset, I switched to the rsClub market with hundred thousand web data. Also, I implemented Python libraries such as Gensim and modules such as Xlsxwriter and Matplotlib for satisfying data readbility and visualization.

Result (R):

The calculated correlations, approximate -0.0031, of listing similarities and seller levels of all sellers based on processed data indicates that there was no relation between sellers' listings similarities and sellers' levels. The interpretation helped my director and other professors in this field better understand darknet markets.

Please leave comments if you have any question, thanks!