AgriPoliS - Software Management Plan

2023-03-07

1 Introduction

A software management plan (SMP) formalizes the goals and structures of software projects, clarifies roles and responsibilities of the persons involved in these projects, and helps to onboard new team members or external users. The overview that a SMP provides can help new team members to get an understanding of how their actions impact the work of other team members and how they can support one another; assisting in increasing productivity in work processes. Additionally, an SMP enhances software transparency and provides information about the publication strategy and reproducibility. The format of an SMP is flexible and can be tailored to the project or the requirements of the funder. At its core, the SMP should focus on the expected software outputs, their management and assurance throughout the project stages.

AgriPoliS (Agricultural Policy Simulator) is an agent-based model based on the work of Balmann (1997) and since Happe (2004), has been developed to depict real agricultural regions. Today, the model’s development relies largely on third-party funded projects. Over recent years there have been several projects related to the development and extension of AgriPoliS in different ways. Despite the differences, every project adheres to core software development principles. This document aims to outline those principles.

This SMP will provide an overview of the agent-based model AgriPoliS, references to documentation and information about how the software development is organized, with a focus on workflow and responsibilities, and the revision and control process during the software development. This document will be continuously updated as AgriPoliS is further developed.

This document is devided into five sections:

2 Scope, purpose, and objectives

AgriPoliS is an agent-based model used to study agricultural structural change in EU agriculture. It can be calibrated to empirical data derived from farm accounting data and regional statistics, and therefore can model actual regions. In AgriPoliS, farms are represented as agents, whose objective is to maximize profits (in the case of corporate farms) or household income (in the case of family farms). This is done by the use of a mixed-integer programing model, which is linked to the selected farm agent’s data on factor endowments, such as stables and machinery, labor, capital, land, and managerial ability, as well as the various production and investment alternatives available to them. Each farm decides what to produce, how much to produce, and whether it is worth renting additional land or rather releasing some of it onto the land market. In addition, each farm agent has the option to exit agriculture. Depending on their individual situation and what is happening on the neighboring farms, farms react differently. The heterogeneity in the farm population enables the model to cover these individual responses.

In agent-based models, interactions between agents play a central role. In the case of AgriPoliS, the interactions occur on the land market. The dynamics on land markets are mainly determined by the interactions between individual farms, hence, agent-based approaches offer advantages for creating an explicit land market. In AgriPoliS, the land allocation process is implemented as an iterative auction where available plots of land are allocated to farm agents seeking to rent additional land. During an iterative auction, bids from different farms for a particular plot of land are compared, and the agent with the highest bid receives the plot. This process is repeated in subsequent rounds until either all available plots have been allocated or no more bids are made.

For the analysis, AgriPoliS enables ex-post and ex-ante studies of agricultural structural change, particularly regarding the impact of alternative policies by comparing actual policies with counterfactual scenarios. Effects of alternative scenarios can be analyzed on several levels. These include individual behavior (e.g. regarding investments) and the overall performance (e.g. profits, liquidity, size) of individual farms but also those of a specific group of farms as well as number of farms, cultivation patterns, and the land market of entire agricultural regions. Based on these outputs, it is possible to interpret the results with respect to production activities and intensities, economic performance of farms, income distribution, farm structure and environmental impacts. The results can be used to derive policy recommendations targeted to specific policy goals and regions.

3 Reference documents

Information in the style of a handbook can be found on the AgriPoliS portal. The AgriPoliS Manual provides necessary information for AgriPoliS users at all levels and focuses on application-oriented descriptions of the model and its components. It can be accessed via the following link:

AgriPoliS Manual

AgriPoliS-RDM is the research data management system for AgriPoliS and provides datasets, publications, scripts etc. It also provides the link to the source code which is hosted on GitHub. Access under AgriPoliS2020

Several further documents provide insights into the structure and application of AgriPoliS. All documents listed below can be accessed via: AgriPoliS Documentation

Detailed and extensive model documentation are provided by Kellermann et al. (2008) and Happe (2004). The later also elaborates on how AgriPoliS can be used to study agricultural policies and their effects on structural change in agriculture and provides a documentation of the adaptation of AgriPoliS to the region Hohenlohe.
An introduction to AgriPoliS following the ODD-protocol (Overview, Design concepts, and Details) by Sahrbacher et al. (2012) provides the description of the model purpose, the entities and a process overview. Further, it provides information about the basic applied principles and the initialization process.
A description of how to adapt AgriPoliS to a region was created by Sahrbacher and Happe (2008). They focus on the collection and compilation of input files for AgriPoliS and describe the development of a virtual farm structure, as well as the steps of modeling the farm agents.

4 Software development activities and responsibilities

As mentioned in the introduction, each project in which AgriPoliS is involved has its own timelines, objectives, and research outcomes. This requires creating a project-specific software management plan for each new project, where the software development is tailored to the project’s schedule and requirements (see Table 2). The following draft outline is therefore general and can be flexibly adapted to the requirements of the respective project by the AgriPoliS team. The AgriPoliS team consists of the PI, programmer, and researchers who meet regularly to discuss current and future projects involving AgriPoliS as well as specific model adaptations, progress, and further AgriPoliS issues. Typically, the team meets once a month with a designated team member responsible for organizing and moderating the meeting. Following each meeting, the organizer for the next meeting is selected. The following components of a project and their corresponding responsibilities can be traced in Table 1.

Table 1: Software activities and responsibilities

Usually, a project begins with a planning phase where the Principal Investigator (PI), in collaboration with other researchers such as project partners or prospective PhD students conducts a problem assessment, defines research objectives, and determines time requirements. At this stage, the PI/researcher formulates the research proposal and subdivides it into work packages and tasks. Depending on the size of the project, work packages may have different individuals responsible for them. To ensure efficient software development, tasks, milestones, and deadlines are established based on the overall project plan. The project proposal and software development-related elements are preferably shared with the programmer well in advance of the project's start to allow for early coordination of the project work with other programming tasks.

Once a project gets funding, the researcher (often PhD student), in coordination with the PI, plans the necessary adaptations of AgriPoliS. It is recommended to discuss this concept with the AgriPoliS team before proceeding with the actual implementation. Once the concept is ready, the researcher provides the programmer with a detailed description of the required code changes, ideally in a flowchart with pseudocode. At that point it is expected that the researcher and the programmer agree on a timeline. The programmer is responsible for making changes to the code as well as checking and (if necessary) debugging of the code. When the code is successfully written, the new model version is handed over to the researcher who will further verify and validate the code changes according to the testing strategy mentioned in section 4 of the revision control process (see section 5). At this point, the programmer and researcher work closely together and coordinate their work so that problems and ambiguities can be resolved quickly. There may also be deviations from the expectations in the proposal, e.g., if the intended modeling turns out to be too complicated to fulfill in the time schedule. To finalize the validation, the persons involved (usually researcher, programmer, and PI) have to agree on a model version that is ready to be used in further steps of the project. This usually involves at least one AgriPoliS team meeting where results of the validation are discussed by the whole AgriPoliS team. The points described so far are usually worked through in an early phase of the project. However, they may well be repeated if it becomes apparent in the further course of the project that further adjustments to the model are required. Researchers can then proceed with simulations and analysis of the different scenarios, and then evaluate the results with the PI and/or the whole AgriPoliS team.

Throughout the development of the project phases, it is the responsibility of the PI to monitor the progress, and to coordinate team meetings, updates, and deliverables. It is however also the responsibility of the WP leaders and researchers to provide updates, propose and prepare topics for the team meetings, and keep the schedule for deliverables. Finalizing work packages and the whole project in the end is an effort where the entire team is involved.

The following example of a project schedule for AgriPoliS outlines how a project might be structured.

Work packages and tasks

Milestones

Software related activities

Table 2: Schedule of project and related software activities

5 Revision control process

Implementing new functionalities, policies, or regions in AgriPoliS requires a control process that ensures the quality and robustness of the adaptation. Because these are crucial processes, they will be described in detail.

5.1 Terminology

Verification is the process of determining if a simulation program works as intended (e.g., debugging) (Xiang et al. 2005; Ormerod and Rosewell 2009; Niazi et al. 2017). Validation is the process of determining if the computational model is an accurate representation of the actual system that is being analyzed (Xiang et al. 2005; Ormerod and Rosewell 2009; Niazi et al. 2017). Both go hand in hand and are often conducted simultaneously. Figure 1 provides a schematic presentation of these interrelated processes.

Figure 1: Visualization of the verification and validation process (own visualization based on Standridge (2021))

The process of creating a conceptual model typically involves examining real-world systems, such as agricultural regions and their patterns of change. AgriPoliS is based on the conceptualization of agricultural systems as complex adaptive systems. It is crucial to understand the system's functions, such as how changes in agriculture's structure arise from interactions between individual farmers and agricultural policies. Balmann (1997) developed the initial model, which formed the basis for subsequent AgriPoliS versions. The name AgriPoliS was first used by Happe and Balmann (2002). Over its more than 20-year existence, AgriPoliS has undergone thorough validation through numerous applications in various research projects, peer reviews, and scientific debates. Therefore, we consider the conceptual model to be validated. The following sections will focus on the verification and validation of AgriPoliS adaptations and extensions.

The general work process for AgriPoliS adaptations includes the steps implementation, verification by programmer (compilation and debugging of code), verification by researcher, and validation.

5.2 Implementation

The process of adapting a model can vary depending on the characteristics of the adaptation and may have different requirements. In the case of AgriPoliS, we typically distinguish between two types of model adaptation that require different revision control processes: the implementation of a new region or new functionalities.

5.2.1 Implementation of a new region

AgriPoliS is calibrated to various European regions (see regions), and implementing a new region can typically be done without making changes to the AgriPoliS source code. When implementing a new region, it is crucial to maintain the structure of input files (input file directory ). To simplify the process, it may be helpful to use the folder structure of an existing region and make modifications as necessary.

Detailed instructions on how to adapt AgriPoliS to a new region are available in the AgriPoliS Manual. Generally, these steps do not require any particular programming skills, although a good understanding of AgriPoliS and its input file structure is necessary.

5.2.2 Implementation of model functionalities

If new functions are being added or existing ones are being modified, the source code usually requires changes. As not all researchers have the necessary programming skills, it is usually the programmer’s task to implement these adaptations. However, it is the responsibility of the researcher to provide the programmer with a detailed description of the required code changes.

5.3 Verification

Verification tasks are divided between the programmer and the researcher. The programmer is responsible for verifying the source code through compilation and debugging, while the researcher verifies if new implementations work as intended. To reduce computing time of verification, it is recommended to increase the "Teiler" (found under inputfiles/options) and run simulations without repetitions. Typically, a "Teiler" between 3 to 10 is used to simulate only a fraction (1/3 to 1/10) of the region's farms.

To ensure the accuracy of any changes made to AgriPoliS, it is important to establish a reference scenario for the new implementation. This involves defining both the AgriPoliS version and the region to be used as reference for subsequent verification. One option of checking the correct implementation of new functionalities is by deactivating new functionalities (setting newly added variables to zero) and matching these simulation results to the reference scenario. Any discrepancies indicate a faulty implementation that requires investigation to determine the source of the error. When implementing a new model region, the focus is on verifying the structure of the input files, with potential errors arising from formatting, farm compilation, and inconsistencies in file designations.

The verification of the correct implementation is followed by outcome testing to verify if the new version works as intended.

5.3.1 Outcome testing

AgriPoliS generates output files that provide a wide range of results covering various aspects, including production, farm economics, production intensity, income distribution, and farm structure (see href="https://www.agripolis.org/agripolis/manual#outputpos">AgriPoliS Outputs). The researcher needs to select a set of key indicators from these results to evaluate the reliability of the outcomes. To verify the outcomes, certain indicators are particularly useful, such as the number of farms, the number of farms in specific farm types, production patterns, livestock numbers, investments, and profits. Some of these indicators are generally useful across regions, while others may be region-specific or related to a particular production activity. For instance, the number of dairy cows may be a relevant indicator for a region with a significant dairy production sector.

In certain instances, particular indicators can serve as clear evidence of input falsification, such as a sudden reduction in the number of farms or a specific livestock species after a few simulation periods. However, for other cases, evaluating the plausibility of indicators may not be straightforward and may necessitate a thorough comprehension of the case study region. Critical components of the input files, such as market prices, production costs, opportunity costs, management factors, and investments, can significantly impact the outcomes and require close scrutiny. In such cases, consulting with the PI, the programmer, and the entire AgriPoliS team can be useful in devising an effective approach.

5.4 Validation

Effective coordination and communication among researchers, programmers, and PI are crucial for interrelated verification and validation processes. After verifying the plausibility of key outcome indicators, the validation of simulations and scenarios can proceed. Typically, this involves comparing and validating the simulation results with real-world systems. However, AgriPoliS simulations often involve hypothetical or counterfactual scenarios, which can make it challenging to validate due to insufficient data quality or availability. To overcome these challenges, it is necessary to involve experts and adopt specific approaches tailored to the agricultural, economic, and region-specific context (see below).

When simulation results are invalid (which is not unusual), it is necessary to refine the implementation and repeat the process until the model successfully passes all tests and is considered validated. This iterative refinement and testing process is crucial to ensure that AgriPoliS functions as intended and is reliable.

For validation, different context specific approaches are possible and can complement each other:

a) Empirical data comparison:

Because AgriPoliS often simulates real regions, it is possible to validate results with empirical regional data, depending on availability. An empirical data comparison is used in particular to check the accuracy of the implementation of a new region by comparing simulated key figures with available regional data, e.g. from the farm structural survey. The extent to which deviations are acceptable depends on the specific application and question at hand. However, validating the model outcome through this method can be challenging as data quality or availability may be limited.

An empirical data comparison for AgriPoliS was for instance performed for the Emsland region to validate simulation results related to farm structures over the period 2015 – 2020. This example can be found in the appendix.

b) Stakeholder participation

As mentioned above, a good understanding of the case study region is crucial for assessing the plausibility of the simulation results. It is therefore desirable to involve regional experts in the validation process whenever possible. This participatory approach, as noted by Whitmarsh and Nykvist (2008), can lead to a higher acceptance of the simulation results in the public and for policy makers.

For validation in AgriPoliS, we conduct stakeholder workshops where individuals with specific roles in the agricultural system are invited to systematically discuss and validate simulation results. This method is particularly effective in evaluating proposed or hypothetical policies. In instances where data is insufficient, we complement the empirical parameterization of the model with focus groups or semi-structured expert interviews.

Examples of stakeholder participation in AgriPoliS can be found in Ostermeyer & Balmann (2011), Pitson et. al (2020a).

c) Sensitivity analysis:

Sensitivity analysis is a technique that is used to examine how variations in the model input affect the model output and can help to identify the relative importance of inputs. It is conducted by changing an input variable and observe how the output variables respond. The purpose of sensitivity analysis is to identify which input variables have the most significant impact on the output and determine the extent to which changes in each input variable influence the output variable. In contrast to traditional sensitivity analysis that relies on changing only one input parameter at a time, AgriPoliS, as an agent-based model, has the advantage that the interactions between input factors can be studied. Therefore, the preferred approach for sensitivity analysis for AgriPoliS is to simultaneously vary several input factors in a structured way. Such a Design of Experiments (DOE) provides a systematic way to carry out simulation experiments, taking into account parameter interactions, and enables the investigation of specific aspects of a simulation model while incorporating statistical aspects into the analysis of results. An example of a sensitivity analysis for AgriPoliS can be found in Happe (2004, p. 92 ff).

d) Behavioral experiments FarmAgriPoliS

To validate behavioral assumptions in AgriPoliS, behavioral experiments were conducted with FarmAgriPoliS. FarmAgriPoliS is a business management game based on AgriPoliS. In FarmAgriPoliS, a human player assumes the role of a farm manager and competes against other computer-simulated farms. FarmAgriPoliS can provide insights on risk considerations, strategic behavior, and possibly social attitudes and mental models. The human decisions can then be compared to the decisions of a computer agent, and therefore provides an opportunity for testing and validating the behavioral assumptions in AgriPoliS. Appel and Balmann (2019) found that the behavior of human players and computer agents differs significantly, in a way that humans are better in avoiding losses and worse in realizing high profits. FarmAgriPoliS can be adapted to specific decision-making situations, e.g., contribution to (collective) agri-environmental measures.

e) Colleagues and the research community:

Furthermore, regular exchange within the scientific community is an important aspect that contributes to the continuous quality control of the model. This includes research group meetings and scientific conferences, where researchers from the AgriPoliS team present novelties to the model and their simulation results. It is particularly useful to present this to researchers working on simulation models or related fields of agricultural economics to gather critical (and hopefully constructive) feedback.

6 License, intellectual property

AgriPoliS is a software developed at Leibniz Institute of Agricultural Development in Transition Economies (IAMO). It is published under the MIT License on GitHub: https://github.com/agripolis/agripolis2020

The contributors to the development of AgriPoliS are listed in the AUTHORS file on GitHub.

A data management plan is available at: https://www.agripolis.org/agripolis/datamanagement  

Appendix

Example for empirical data comparison

We compare the simulation results to data from the region that was collected by the regional expert A. Krämer (Landwirtschaftskammer (Chamber of agriculture) Emsland). Krämer provided a document of the development of the agricultural structure of the Emsland from the years 2015 - 2020. The document contains information about farm sizes and attempts to gather data on farm split-ups. Krämer's work is particularly useful for validating the first few years of our simulations, as the initialization year in our simulations is 2016.

When looking at the agricultural structure of the Emsland in the data collection by A. Krämer and our modeling for AgriPoliS, some differences become apparent, which are due to the type of survey. In his report, Krämer records all farms that have submitted an agricultural application. He also attempts to identify farms that work in family or part-family networks. In these cases, there are two or more applicants, but they can be considered as one household or farmstead. Reasons for these divisions can be found, for example, in tax and subsidy law. It determines these related companies by recording e.g., identical addresses or telephone numbers. He summarizes these farms in the statistics as “adjusted.” What it does not cover, however, are commercial operations without land equipment. The associations of agricultural and commercial enterprises are therefore not shown.

For the illustration in AgriPoliS, we chose the farm typology approach. This approach is used to map the heterogeneity of an agricultural region when existing data (e.g., FADN) are not available or insufficient. Using this method, we determined the equipment and characteristics of "typical" farms in the Emsland region in expert discussions at the Emsland Chamber of Agriculture. Using this method, we created 22 typical farms for the Emsland region and then upscaled their individual number to data from the agricultural structure survey. As upscaling indicators, we chose those that are not influenced by farm divisions. Specifically, these are land and livestock numbers. This modeling method implies that we have not recorded the spectrum of small and part-time farms in all their diversity. We recorded farms with their associated commercial businesses as one farm (several legal units but only one management unit), which was the goal for representing the core characteristics of the Emsland.

With the land allocation in the expert’s data, it is noticeable that despite the adjustment, approx. 42% of the farms still farm on less than 20 ha. Except for one part-time farm with a very low weighting, we have not shown these farms at all, which means that there is no coherent picture here. This can be explained as the modeling focus is on typical farms of the regions which act as profit maximizers, while small farms farmed on a part-time basis often have a hobby aspect to them. Therefore, the overall share of land in the region that these farms manage is small and their impact on the region's development is minimal. Conversely, the farms captured in the model have a significant impact on the region's overall development.

References