MarkupAutomator AB Test Repository

This repository contains documents and resources related to the AB test described in our paper.

Overview

The AB test is a method of comparing two versions of a webpage or other user experience to determine which one performs better. In our paper, we describe an AB test that we conducted to evaluate the LLM's markup vs Human markup.

Google Form

You can find our conducted forms in the following links: Important: If you're planning to conduct this AB test, we recommend performing the tests on different days to eliminate any potential bias caused by participants remembering the test.

1. https://forms.gle/gS4MNtE2MrKTwa5c7
2. https://forms.gle/LqTxdUEA9vgzGrcL6

Human Evaluation

You can find the result files according the above forms in the following links:

1. https://docs.google.com/spreadsheets/d/1W0MaY6lbxt5JZckwc5_Btf-nZwe1Z-pagpyq8Qay5Qc/edit?usp=sharing
2. https://docs.google.com/spreadsheets/d/13WiI0DgYq7jY-BOw6Be8bbc9MLNdpF2IVU4kvcR3JdA/edit?usp=sharing