v17 Importing existing trial designs and data

Frequently, especially when you start using BMS, you will have an existing trial which has been designed and planted outside BMS. You may or may not already have data collected for this trial but you want to import the trial design and data (if it exists) into BMS and proceed with analysis from BMS.
Actually in BMS there are two ways to perform this operation, one using the Study manager with a feature to Import your own design, and another using an Information Management application called Import Datasets. We will demonstrate the Study manager approach with Import your own design. For the second method you can follow the tutorial : How to Import Historical Trial Data.

Objectives

At the end of this tutorial, the user should be able to:

  1. Import entries for an existing or historical trial
  2. Create a study for an existing or historical trail
  3. Import the design of an existing or historical trial
  4. Load data for an existing or historical trial.

Creating a germplasm list for an existing trial

Often an existing trial may be available in excel such as the file CGM20AVT.xlsx in Sample Files.zip where the experimental design and the data are stored in a format like the one shown below.

There may be data for several sites stacked together as with the file CGM20AVT.xls (4 sites) which you can find in Sample Files.zip, or the data for different sites may come as separate but similar files. We need to load this data into a BMS study for analysis.
The first thing to do is to create a germplasm list for the distinct entries in the trial. This can be done by finding the entries in the BMS and adding them to a list with the List Manager, or it can be done by extracting the distinct entries from the file(s) and importing them with Germplasm Import.
We will use Germplasm Import, but it is important to check that the entry numbers and the designations are consistent across every rep of the trial. When you have checked this you can extract the ENTRY_NO and DESIGNATION for site 1 rep 1 and paste it into a Germplasm Import template:

The observation sheet looks like this:

And the description sheet like this:





This list is now imported int BMS taking care to select existing entries wherever appropriate: Go to Import Germplasm, browse for the template file, fill in the import details and click Finish.


Select appropriate germplasm when there are multiple hits, such as for the checks IR 64 and IR 72. (Select the ones with seed stock):

Save the list:

Extracting a lay-out file for an existing trial


Since this trial has already been planted, we cannot generate a trial in BMS. Instead we must read in the existing randomization – the lay-out.
A lay-out file is a very simple csv file which contains information about the experimental design and optionally, the trait names and values. It has one row for each plot in the trial and it must have a column called TRIAL_INSTANCE containing an integer number indicating which site the plot comes from, it must have a column called PLOT_NO with a sequence number of 1 to number of plots at each location, it must have a column called ENTRY_NO indicating which entry is planted on the plot. Then it may have columns like REP_NO and BLOCK_NO giving design details, and it can have columns with trait names for headings and these columns may or may not have data. If they have data it will be entered, if they do not, the expectation is that fieldbooks will be exported after the design is completed and the data collected and entered through the fieldbooks in the usual way.


So our lay-out file could look as follows:

It is best have the data for all the sites stacked together in one file, but not essential, extra sites can be added later.

Create the Study for the trial

From the Study Manager select Start a new study. Enter some metadata, and select use and existing study as a template.
On the Germplasm and Checks tab enter the germplasm list we imported and set the last two entries, IR 64 and IR 72 to be check entries. (click on their Entry Type to change it)

Save the new trial in your 2020 Trials folder, make one if you do not have one (CGM 2020 Trials for me).
On the Environments tab, set the number of environments to 4 and click ok. Then in the location names box look for the sites of the trial - Raipur RRS, Titabar RRS, Pusa RRS and Cuttack RRS. If the locations do not exist in your database they need to be added by a Crop Manager since the function of adding locations is a crop role not a program role.

The Environment numbers and names must match those in the original data file and the lay-out file. If you know the plot size and seeding date, enter them.
Go to the Observations sheet and remove all the traits. Select them and click Remove. This is because we will be entering the traits from the lay-out file in this case.

Import the lay-out file

Now open the Experimental Design tab and click import an experimental design:

Browse to the layout file and open it:

Click Continue to read the layout file and you will open a trait mapping form. Nearly all the traits have been automatically mapped to variables in the ontology because they have exactly the same names. You should check each to be sure the automatic mapping is correct. Any variables which cannot be mapped remain in the Unmapped variables box.

In our case GYGP – grain yield in grams per plot has not been mapped. It should be mapped to the variable GrYld_wgh_gplot. To make this mapping drag the variable down and place it in the Traits section then click Apply Mapping.






This will open the Ontology search box. Look for grain yield traits and select GrYld_wgh_gplot.

Now the mapping is complete click Next.
You will get a review panel to check the import:

Click Finish and the design (and data in our case) will be stored.
Add trait GYTHA if necessary and compute its values for all locations.