Things that strengthen an IR paper

Presentation

1. The paper’s motivation and the potential impact of the addressed problem are discussed.
2. The paper’s original contributions (i.e. the delta over prior art) are clearly stated.
3. The paper’s claims are properly scoped and supported.
4. The paper clearly describes what was done and what was not.
5. The choices made in each step of the research are justified (the why’s).
6. The results are presented effectively in appropriate format.
7. Good discussion accompanies the results.

Experimentation (if applicable)

1. The experimental design and its scale are appropriate.
2. In comparative studies, appropriate baselines are used.
3. The experimental results are reliable and generalizable.
4. The evaluation methods employed are in line with the research questions.
5. Statistical analysis is performed and reported appropriately.
6. Sufficient details (with data and code where appropriate) are provided to help other researchers assess and reproduce the experiments.