Evidence Factory - Paper quality evaluation questionnaire

1) How the practice was applied?

It is unclear how the practice was applied.

One experiment (i.e., a trial of the practice out of context).

A series of experiments, or for one pilot project (i.e., a trial of the practice in context).

A series of pilot projects, or for one application in a production project.

Application in a series of production projects.

2) How the results were measured?

It is unclear how the results were measured.

A subjective opinion such as a lesson learned.

A case study (i.e., the results were obtained within a project lifetime), or a loose comparison between practices.

A rigorous comparison with another practice, across projects, or with a baseline.

3) How the evidence was reported?

The evidence was gained via an interview, or if the evidence was web-published as a report (e.g., white paper), expert opinion, or lesson learned.

Publication as a technical report within a university or organization, or as part of workshop proceedings.

Publication in a trade journal (e.g., CrossTalk).

Publication in a conference or most academic journals.

Publication in the highest-quality archival journals.

4) Who reported the evidence?

It is unclear who reported the results.

Someone external to the team that applied the practice (i.e., a person who got the evidence second hand or after the fact).

Someone who was part of the group that applied the practice.

Quality evaluation questionnaire for non-systematic studies.