Skip to content

Latest commit

 

History

History
359 lines (179 loc) · 29.8 KB

Benchmarks.md

File metadata and controls

359 lines (179 loc) · 29.8 KB
Raw

Paper Collection for Code-related Evaluation and Benchmarks

  1. [Preprint] CodeRAG-Bench: Can Retrieval Augment Code Generation? arXiv, 2024.05

    Zora Zhiruo Wang, Akari Asai, Xinyan Velocity Yu, Frank F. Xu, Yiqing Xie, Graham Neubig, Daniel Fried

  2. [Preprint] MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation arXiv, 2024.05

    Jianbo Dai, Jianqiao Lu, Yunlong Feng, Rongju Ruan, Ming Cheng, Haochen Tan, Zhijiang Guo

  3. [Preprint] NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts. arXiv, 2024.05

    Shudan Zhang, Hanlin Zhao, Xiao Liu, Qinkai Zheng, Zehan Qi, Xiaotao Gu, Xiaohan Zhang, Yuxiao Dong, Jie Tang

  4. [Preprint] DebugBench: Evaluating Debugging Capability of Large Language Models. arXiv, 2024.01

    Runchu Tian, Yining Ye, Yujia Qin, Xin Cong, Yankai Lin, Yinxu Pan, Yesai Wu, Zhiyuan Liu, Maosong Sun

  5. [Preprint] VulBench How Far Have We Gone in Vulnerability Detection Using Large Language Models. arXiv, 2023.12

    Zeyu Gao, Hao Wang, Yuchen Zhou, Wenyu Zhu, Chao Zhang

  6. [Preprint] InstructCoder InstructCoder: Empowering Language Models for Code Editing. arXiv, 2023.10

    Qisheng Hu, Kaixin Li, Xu Zhao, Yuxi Xie, Tiedong Liu, Hui Chen, Qizhe Xie, Junxian He

  7. [Preprint] EvalGPTFix A Critical Review of Large Language Model on Software Engineering: An Example from ChatGPT and Automated Program Repair. arXiv, 2023.10

    Quanjun Zhang, Tongke Zhang, Juan Zhai, Chunrong Fang, Bowen Yu, Weisong Sun, Zhenyu Chen

  8. [Preprint] SWE-bench SWE-bench: Can Language Models Resolve Real-World GitHub Issues? arXiv, 2023.10

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan

  9. [EMNLP2023] CodeTransOcean CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation. arXiv, 2023.10

    Weixiang Yan, Yuchen Tian, Yunzhe Li, Qian Chen, Wen Wang

  10. [Preprint] CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models. arXiv, 2023.09

    Lingyue Fu, Huacan Chai, Shuang Luo, Kounianhua Du, Weiming Zhang, Longteng Fan, Jiayi Lei, Renting Rui, Jianghao Lin, Yuchen Fang, Yifan Liu, Jingkuan Wang, Siyuan Qi, Kangning Zhang, Weinan Zhang, Yong Yu

  11. [Preprint] VerilogEval: Evaluating Large Language Models for Verilog Code Generation. arXiv, 2023.09

Mingjie Liu, Nathaniel Pinckney, Brucek Khailany, Haoxing Ren

  1. [ASE2023] G-TransEval On the Evaluation of Neural Code Translation: Taxonomy and Benchmark. arXiv, 2023.08

Mingsheng Jiao, Tingrui Yu, Xuan Li, Guanjie Qiu, Xiaodong Gu, Beijun Shen 12. [ASE2023] CommitChronicle From Commit Message Generation to History-Aware Commit Message Completion. arXiv, 2023.08

Aleksandra Eliseeva, Yaroslav Sokolov, Egor Bogomolov, Yaroslav Golubev, Danny Dig, Timofey Bryksin

  1. [Preprint] HumanEvalPack OctoPack: Instruction Tuning Code Large Language Models. arXiv, 2023.08

Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, Shayne Longpre

  1. [Preprint] LogHub A Large-scale Benchmark for Log Parsing. arXiv, 2023.08

Zhihan Jiang, Jinyang Liu, Junjie Huang, Yichen Li, Yintong Huo, Jiazhen Gu, Zhuangbin Chen, Jieming Zhu, Michael R. Lyu

  1. [ICLR2024] BioCoder BioCoder: A Benchmark for Bioinformatics Code Generation with Contextual Pragmatic Knowledge. arXiv, 2023.08

Xiangru Tang, Bill Qian, Rick Gao, Jiakang Chen, Xinyun Chen, Mark Gerstein

  1. [ASE2023] ExGroFi Delving into Commit-Issue Correlation to Enhance Commit Message Generation Models. arXiv, 2023.07

Liran Wang, Xunzhu Tang, Yichen He, Changyu Ren, Shuhua Shi, Chaoran Yan, Zhoujun Li

  1. [TOSEM] CoRec Context-aware Retrieval-based Deep Commit Message Generation. 2023.07

Haoye Wang, Xin Xia, David Lo, Qiang He, Xinyu Wang, John Grundy

  1. [SIGIR2023] COJ2022 Errorclr: Semantic error classification, localization and repair for introductory programming assignments. 2023.07

Siqi Han, Yu Wang, Xuesong Lu

  1. [Preprint] VulnPatchPairs Limits of Machine Learning for Automatic Vulnerability Detection. arXiv, 2023.06

Niklas Risse, Marcel Böhme

  1. [NeurIPS2023] DotPrompts Guiding Language Models of Code with Global Context using Monitors. arXiv, 2023.06

Lakshya A Agrawal, Aditya Kanade, Navin Goyal, Shuvendu K. Lahiri, Sriram K. Rajamani

  1. [ICML2023] LongCoder LongCoder: A Long-Range Pre-trained Language Model for Code Completion. arXiv, 2023.06

Daya Guo, Canwen Xu, Nan Duan, Jian Yin, Julian McAuley

  1. [Preprint] StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code. arXiv, 2023.06

Hannah McLean Babe, Sydney Nguyen, Yangtian Zi, Arjun Guha, Molly Q Feldman, Carolyn Jane Anderson

  1. [Preprint] EvalPlus Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. arXiv, 2023.05

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming Zhang

  1. [RAID2023] DiverseVul DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection. arXiv, 2023.04

Yizheng Chen, Zhoujie Ding, Lamya Alowain, Xinyun Chen, David Wagner

  1. [Preprint] RunBugRun An Executable Dataset for Automated Program Repair. arXiv, 2023.04

Julian Aron Prenner, Romain Robbes

  1. [Preprint] HumanEval-X CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X. arXiv, 2023.03

Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, Jie Tang

  1. [Preprint] xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval. arXiv, 2023.03

Mohammad Abdullah Matin Khan, M Saiful Bari, Xuan Long Do, Weishi Wang, Md Rizwan Parvez, Shafiq Joty

  1. [EMNLP (Findings)2023] ODEX Execution-Based Evaluation for Open-Domain Code Generation. arXiv, 2022.12

    Zhiruo Wang, Shuyan Zhou, Daniel Fried, Graham Neubig

  2. [Preprint] ARCADE Natural Language to Code Generation in Interactive Data Science Notebooks. arXiv, 2022.12

    Pengcheng Yin, Wen-Ding Li, Kefan Xiao, Abhishek Rao, Yeming Wen, Kensen Shi, Joshua Howland, Paige Bailey, Michele Catasta, Henryk Michalewski, Alex Polozov, Charles Sutton

  3. [ICML2023] DS-1000 DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation. arXiv, 2022.11

    Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Scott Wen-tau Yih, Daniel Fried, Sida Wang, Tao Yu

  4. [DaSH2022] ExeDS Execution-based Evaluation for Data Science Code Generation Models. arXiv, 2022.11

    Junjie Huang, Chenglong Wang, Jipeng Zhang, Cong Yan, Haotian Cui, Jeevana Priya Inala, Colin Clement, Nan Duan, Jianfeng Gao

  5. [EMNLP2022Findings] TorchDataEval When Language Model Meets Private Library. arXiv, 2022.10

    Daoguang Zan, Bei Chen, Zeqi Lin, Bei Guan, Yongji Wang, Jian-Guang Lou

  6. [ICLR2023] MBXP, Multilingual HumanEval, MathQA-X Multi-lingual Evaluation of Code Generation Models. arXiv, 2022.10

    Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang

  7. [TSE2023] MultiPL-E: a scalable and polyglot approach to benchmarking neural code generation. arXiv, 2022.08

Federico Cassano , John Gouwar , Daniel Nguyen , Sydney Nguyen , Luna Phipps-Costin , Donald Pinckney , Ming-Ho Yee , Yangtian Zi , Carolyn Jane Anderson , Molly Q Feldman , Arjun Guha , Michael Greenberg , Abhinav Jangda

  1. [Preprint] AixBench: A Code Generation Benchmark Dataset. arXiv, 2022.06

    Yiyang Hao, Ge Li, Yongqiang Liu, Xiaowei Miao, He Zong, Siyuan Jiang, Yang Liu, He Wei

  2. [IJCAI2022] PandasEval, NumpyEval CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation. arXiv, 2022.06

    Daoguang Zan, Bei Chen, Dejian Yang, Zeqi Lin, Minsu Kim, Bei Guan, Yongji Wang, Weizhu Chen, Jian-Guang Lou

  3. [TMLR2023] BIG-Bench Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. arXiv, 2022.06

    BIG-Bench Authors

  4. [EACL2023] MCoNaLa MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages. arXiv, 2022.05

    Zhiruo Wang, Grace Cuenca, Shuyan Zhou, Frank F. Xu, Graham Neubig

  5. [AAAI2022] CoST Multilingual Code Snippets Training for Program Translation. 2022.06

    Ming Zhu, Karthik Suresh, Chandan K Reddy

  6. [ICLR2023] MTPB CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. arXiv, 2022.03

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, Caiming Xiong

  7. [ACL2023] MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages. arXiv, 2022.03

    Zhiruo Wang, Grace Cuenca, Shuyan Zhou, Frank F. Xu, Graham Neubig

  8. [Preprint] CodeContests Competition-Level Code Generation with AlphaCode. arXiv, 2022.02

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, Oriol Vinyals

  9. [Preprint] DSP Training and Evaluating a Jupyter Notebook Data Science Assistant. arXiv, 2022.01

    Shubham Chandel, Colin B. Clement, Guillermo Serrato, Neel Sundaresan

  10. [Preprint] MBPP Program Synthesis with Large Language Models. arXiv, 2021.08

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, Charles Sutton

  11. [ACL2021] PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context 2021.08

    Xinyun Chen, Linyuan Gong, Alvin Cheung, Dawn Song

  12. [ESEC/FSE2021] CrossVul CrossVul: a cross-language vulnerability dataset with commit data. 2021.08

    Georgios Nikitopoulos, Konstantina Dritsa, Panos Louridas, Dimitris Mitropoulos

  13. [Preprint] HumanEval Evaluating Large Language Models Trained on Code. arXiv, 2021.07

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, Wojciech Zaremba

  14. [ACL2021] KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers. arXiv, 2021.06

    Chia-Hsuan Lee, Oleksandr Polozov, Matthew Richardson

  15. [ACL2021] CommitBERT CommitBERT: Commit Message Generation Using Pre-Trained Programming Language Model. arXiv, 2021.05

    Tae-Hwan Jung

  16. [ACL2021] CoSQA CoSQA: 20,000+ Web Queries for Code Search and Question Answering. arXiv, 2021.05

    Junjie Huang, Duyu Tang, Linjun Shou, Ming Gong, Ke Xu, Daxin Jiang, Ming Zhou, Nan Duan

  17. [NeurIPS2021] APPS Measuring Coding Challenge Competence With APPS. arXiv, 2021.05

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, Jacob Steinhardt

  18. [Preprint] CodeTrans CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing. arXiv, 2021.04

    Ahmed Elnaggar, Wei Ding, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Silvia Severini, Florian Matthes, Burkhard Rost

  19. [NeurIPS2021]CodeXGLUE CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. arXiv, 2021.02

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, Shujie Liu

  20. [EMNLP2020] squall On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries. 2020.10

    Tianze Shi, Chen Zhao, Jordan Boyd-Graber, Hal Daumé III, Lillian Lee

  21. [TKDD] FB-Java Deep Graph Matching and Searching for Semantic Code Retrieval. 2020.10

    Xiang Ling, Lingfei Wu, Saizhuo Wang, Gaoning Pan, Tengfei Ma, Fangli Xu, Alex X. Liu, Chunming Wu, Shouling Ji

  22. [Preprint] SO-DS Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intent. arXiv, 2020.08

    Geert Heyman, Tom Van Cutsem

  23. [ICLR2020] GREAT Global Relational Models of Source Code. 2019.12

    Vincent J. Hellendoorn, Charles Sutton, Rishabh Singh, Petros Maniatis, David Bieber

  24. [TSE] ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking. arXiv, 2019.12

    Shangqing Liu, Cuiyun Gao, Sen Chen, Lun Yiu Nie, Yang Liu

  25. [ASE2019] CLCDSA CLCDSA: Cross Language Code Clone Detection using Syntactical Features and API Documentation. 2019.11

    Kawser Wazed Nafi, Tonny Shekha Kar, Banani Roy, Chanchal K. Roy, Kevin A. Schneider

  26. [EMNLP-IJCNLP2019] JuICe JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation. arXiv, 2019.10

    Rajas Agashe, Srinivasan Iyer, Luke Zettlemoyer

  27. [Preprint] CodeSearchNet CodeSearchNet Challenge: Evaluating the State of Semantic Code Search. arXiv, 2019.09

    Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, Marc Brockschmidt

  28. [NeurIPS2019] Devign Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks. arXiv, 2019.09

    Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, Yang Liu

  29. [EMNLP2019] CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases. arXiv, 2019.09

    Tao Yu, Rui Zhang, He Yang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter S Lasecki, Dragomir Radev

  30. [IJICAI2019] CoDiSum Commit Message Generation for Source Code Changes. 2019.06

    Shengbin Xu, Yuan Yao, Feng Xu, Tianxiao Gu, Hanghang Tong, Jian Lu

  31. [ACL2019] SParC: Cross-Domain Semantic Parsing in Context. arXiv, 2019.06

    Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev

  32. [MSR2019] PtrGNCMsg Generating commit messages from diffs using pointer-generator network. 2019.05

    Qin Liu, Zihe Liu, Hongming Zhu, Hongfei Fan, Bowen Du, Yu Qian

  33. [TOSEM2019] Bugs2Fix An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation. arXiv, 2018.12

    Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, Denys Poshyvanyk

  34. [EMNLP2018] Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. arXiv, 2018.09

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, Dragomir Radev

  35. [TDSC2022] SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities. arXiv, 2018.08

    Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, Zhaoxuan Chen

  36. [EMNLP2018] CONCODE Mapping Language to Code in Programmatic Context. arXiv, 2018.08

    Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Luke Zettlemoyer

  37. [IJCAI2018] TL-CodeSum Summarizing source code with transferred API knowledge. 2018.07

    Xing Hu, Ge Li, Xin Xia, David Lo, Shuai Lu, Zhi Jin

  38. [ICMLA2018] Draper VDISC Automated Vulnerability Detection in Source Code Using Deep Representation Learning. arXiv, 2018.07

    Rebecca L. Russell, Louis Kim, Lei H. Hamilton, Tomo Lazovich, Jacob A. Harer, Onur Ozdemir, Paul M. Ellingwood, Marc W. McConley

  39. [ICPC2018] NL2Bash Deep code comment generation. 2018.05

    Xing Hu, Ge Li, Xin Xia, David Lo, Zhi Jin

  40. [LREC2018] DeepCom NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System. arXiv, 2018.02

    Xi Victoria Lin, Chenglong Wang, Luke Zettlemoyer, Michael D. Ernst

  41. [NDSS2018] CGD VulDeePecker: A Deep Learning-Based System for Vulnerability Detection. arXiv, 2018.01

    Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, Yuyi Zhong

  42. [SPLASH Companion 2017] QuixBugs QuixBugs: a multi-lingual program repair benchmark set based on the quixey challenge. 2017.10

    Derrick Lin, James Koppel, Angela Chen, Armando Solar-Lezama

  43. [Preprint] WikiSQL Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. arXiv, 2017.08

    Victor Zhong, Caiming Xiong, Richard Socher

  44. [ACL2017] CommitGen A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes. arXiv, 2017.04

    Pablo Loyola, Edison Marrese-Taylor, Yutaka Matsuo

  45. [AAAI2017] DeepFix DeepFix: Fixing Common C Language Errors by Deep Learning, 2017.02

    Rahul Gupta, Soham Pal, Aditya Kanade, Shirish Shevade

  46. [SIGPLAN Notices2016] PY150 Probabilistic Model for Code with Decision Trees. 2016.10

    Veselin Raychev, Pavol Bielik, Martin Vechev

  47. [ACL2016] CODE-NN Summarizing Source Code using a Neural Attention Model. 2016.08

    Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Luke Zettlemoyer

  48. [ICSME2015] BigCloneBench Evaluating clone detection tools with BigCloneBench. 2015.10

    Jeffrey Svajlenko, Chanchal K. Roy

  49. [ACL2015] WikiTQ Compositional Semantic Parsing on Semi-Structured Tables. arXiv, 2015.08

    Panupong Pasupat, Percy Liang

  50. [AAAI2016] POJ-104 Convolutional Neural Networks over Tree Structures for Programming Language Processing. arXiv, 2014.09

    Lili Mou, Ge Li, Lu Zhang, Tao Wang, Zhi Jin

  51. [ISSTA2014] Defects4J Defects4J: a database of existing faults to enable controlled testing studies for Java programs. 2014.07

    René Just, Darioush Jalali, Michael D. Ernst

  52. [MSR2013] GitHub Java Corpus Mining Source Code Repositories at Massive Scale using Language Modeling. 2013.05

    Miltiadis Allamanis, Charles Sutton

  53. [HLT1994] ATIS Expanding the Scope of the ATIS Task: The ATIS-3 Corpus. 1994.03

    Deborah A. Dahl, Madeleine Bates, Michael Brown, William Fisher, Kate Hunicke-Smith, David Pallett, Christine Pao, Alexander Rudnicky, Elizabeth Shriberg