Reviewing Proposals and the Evaluation Criteria for Collaborative Research

Reviewing Proposals and the Evaluation Criteria

Type G

<1> Check by beamline scientists

Beamline scientists mainly check the technical feasibility and capability of the experimental group, and their opinions are sent to the referees for the reference.

<2> Preliminary review by referees

Three referees selected for each proposal perform screening of documents based on the chart as below.
The criteria revised:
*In April 2020, we clearly stated an item regarding "technical value" in [1], and [Overall evaluation] was deleted.
*In September 2024, the evaluation criteria for [1] "Academic and technical value" and for [2] "Experimental feasibility" were revised.

[1] Academic and technical value (Overall review in terms both points of view)

Score	Criteria
5	An extremely significant proposal, which is expected to obtain exceptional view or new perspectives, develop other experimental methods.
4	A significant proposal, which is expected to obtain excellent view or develop other effective experimental methods (or introduce/spread).
3	An appropriate proposal, which is expected to have valuable results.
2	A proposal which is not worth performing.
1	A proposal with no value.

[2] Experimental feasibility

Evaluation	Criteria
A	It is well considered and most likely to obtain expected results.
B	It is generally considered and requires further consideration for details.
C	It lacks consideration, and it is highly likely that they can not obtain expected results.
N/A	Unassessable

Evaluation	Criteria
A	An excellent group which is able to deal with troubles somehow by themselves and carry out the experiments.
B	A group which can carry out the experiments if the trouble doesn't happen.
C	A group which needs some help from PF staff to carry out the experiments.
N/A	Unassessable

<3> Overall review by PF-PAC (Photon Factory Program Advisory Committee)

Scoring and the judgement to accept or reject are determined based on the evaluation and review comments by referees, comments by PF staff in charge of safety.
We consider academic and technical value are very important when scoring the proposal.
We consider the relevancy to the whole experimental plan as necessary.
In case a spokesperson has three or more approved proposals (except for Type P) whose validity periods were expired between 1 year and 6 years ago from the first day when the proposal becomes active.
1. Spokespersons who haven't registered their publications more than half of the approved proposals are asked the reason why they have not done yet.
2. According to the reply, PF-PAC analyzes the reasons and considers the score.
3. Proposals without any reply to the inquiry are rejected.

<4> Final determination by Advisory Committee for Institute of Materials Structure Science

After being approved at Advisory Committee for Institute of Materials Structure Science, Director of Institute of Materials Structure Science (IMSS), KEK determines the score and whether to accept or reject.

Type P, U, T, and S2

<1> Check by beamline scientists

The process is same as Type G.

<2> Preliminary review by referees

Screening of documents takes place for Type P by 2 referees, for Type U by 3 referees.
An interview takes place by 3 referees for Type T, 5 or more referees for Type S2.
The process is same as Type G, but the evaluation is carried out considering the characteristics of the proposal.

<3> Overall review by PF-PAC (only for Type T and Type S2)

The process is same as Type G.

<4> Final determination by Advisory Committee for Institute of Materials Structure Science (only for Type T and Type S2)

The process is same as Type G.

*PF-PAC chairperson determines to accept or reject, beamtime allocation for Type P, to accept or reject, and scoring for Type U. The determination will be reported to PF-PAC and Advisory Committee for Institute of Materials Structure Science.

Note: The criteria for review (in Japanese) is a document approved by the PF-PAC. The English version of the criteria (this document) was translated into English based on the Japanese version by the PF-PAC secretariat.

Last modified: 2024-09-30 (English version)