Published January 22, 2021

In late December, the Journal of Food Protection published a study from the University of Houston, claiming to have found safety issues with produce grown by small farmers in Texas.

That study was soon retracted due to its non-compliance with Institutional Review Board standards. FARFA Executive Director Judith McGeary submitted the following letter to the Journal, asking that the publication institute a process to ensure the accuracy of future studies relating to small farms.

The text of the letter is below, or you can view the PDF version here.

January 19, 2021

Didi Loynachan
Administrative Editor
Journal of Food Protection

Dear Dr. Loynachan:

I recognize that the article entitled “Assessing Food Safety Practices Among Texas Small Growers,” which the Journal published online in December, has already been retracted for failure to comply with IRB requirements.  But on behalf of both small farmers and the consumers who rely upon them for food, I remain concerned that the article should not have been published in the first place. I urge the editors and staff of your Journal to consider the numerous inaccurate statements made in the article and take steps to ensure that future studies on food safety on small farms are more thoroughly reviewed before publication.

Many of the article’s conclusions were not supported by the data presented, which came from a poorly constructed survey of an extremely limited group of individuals not representative of Texas small farms.  The article was also littered with incorrect factual statements and inaccurate references.

The article began by mis-stating the definition of small growers.  It stated: “Small growers earn ≤$25,000 in annual sales over a 3-year period and have an average food sale of less than $500,000.”  Beyond being incorrect, this sentence does not even make sense, as it states that small growers may “have an average food sale” of up to 20 times their total “annual sales.”  Presumably, the authors confused the two separate definitions for “not-covered” and “qualified exempt” farms in the Food Safety Modernization Act; not-covered farms sell less than $25,000 of produce annually, while qualified exempt farms sell less than $500,000 of all foods annually and more than half to “qualified end users.” This error demonstrates a fundamental lack of understanding with respect to the basic regulatory framework the article critiques.

The article also cited sources that don’t support the claims being made. For example, in a series of inaccurate statements in its second paragraph, the article states that “[u]sually, growers who are starting as small farmers do not have a background in agriculture.” Setting aside the article’s own finding that more than 80% of the respondents to its survey had at least five years of farming experience, the source that the article cites uses the term “often” rather than “usually” (and in turn does not provide a source for its claim). Next, citing to the same source, the article states that “[t]his lack of previous agricultural knowledge is accompanied with a small or limited income, and no previous food safety training or resources.” The source does not support this claim. And lastly, the article states that “[r]esearch has also shown that small farm growers lack the required resources to implement food safety practices.” The cited source, though, states that “[s]ome farmers lack the monetary resources for third-party audits and employees needed for good agricultural practices certification”; lacking the funding needed to pay for GAP certification is not the same as failing to implement food safety practices.

While these errors (and the incomplete sentences and grammatical errors throughout the article) could be corrected, the article and its underlying survey have other, fundamental problems that cannot be remedied.

First, although the article made numerous conclusions about the practices of small produce growers, the population surveyed was not representative of Texas small produce growers.  Rather, the survey appears to have covered primarily noncommercial growers who have minimal, if any, sales of produce.  The survey contained only a single question about size, asking if the respondent earned more than $50,000 a year in revenue from their farm.  Only three respondents (4.4%) earned above that threshold.  Because the question did not identify any brackets or minimum revenue, there is no way of knowing for certain how many of the respondents had little or no revenue. That seventy-two percent of the respondents sell at a single farmers market less than once a month (and only two of the 70 respondents sold their produce at a farmer’s market on a weekly basis) implies that most respondents are backyard gardeners who occasionally sell their excess products.  Drawing conclusions about small commercial produce growers from this survey is inappropriate.

Moreover, many of the actual conclusions drawn by the article were unsupported by meaningful data; many of the questions used in the survey were so broad as to be useless in assessing food safety issues. For example, the survey asked if the individual used “soil amendments”—and the article suggested that soil amendments could be contaminated—but “soil amendments” could include not only manure or compost but also sand, soft rock phosphate, lime, gypsum, perlite, or any number of other amendments that pose no risk to food safety no matter how they are used. 

Even where the questions covered a topic that could have food safety implications, the survey failed to elicit sufficient data to support its dramatic negative conclusions.  For example:

• Manure Use: The article claimed that there is a “high risk of microbial contamination” on produce grown by the 34% of the respondents who use animal manure. Yet the survey did not ask for any information about how the respondent uses manure.  The Food Safety Modernization Act (FSMA) regulations allow the use of manure so long as it is applied in a way that does not contact produce during application and it either has no contact after application or the potential contact is minimized after application.  It is common for small farmers to till manure into the soil before the crops are even planted, which would meet the FSMA requirements.  Concluding that manure use is a food safety risk with no information about how or when it is applied is inaccurate and misleading.
• Domestic Animals: Again, the article claimed that there is a “high risk of microbial contamination” because of the presence of domestic animals on the premises, saying that having animals on the farm “introduces a risk of fecal contamination onto produce.” Yet “domestic animals,” as used in the survey, includes house cats and dogs.  It also includes domestic livestock who may be housed and cared for at a significant distance from the produce being grown on the farm.  Even large farms that must comply with the full FSMA requirements are allowed to have domestic livestock on the farm, just as they are allowed to use manure.  Yet the article implied that the use of manure or the presence of domestic livestock on small farms create more risks than would be allowed under FSMA.
• Toilet and Hand Washing Stations: The article raised concerns that “39% of the farm owners do not provide handwashing facilities and 46% of farm owners do not provide portable toilet facilities for workers near their farms or packing areas.”  Yet 69.7% of the respondents also indicated that they had only 1 -5 people “working on the farm.” Since the survey asked about anyone working on the farms, not how many employees or non-family workers there were, many of those respondents likely have no one outside the family working on the farm – and the family members would use the toilet and washing facilities in their home.  Even those who have one or two non-family workers (if there even were any such respondents) almost certainly allow them to use the house facilities.  The article states that “the nonavailability of important resources in some farms, such as handwashing and toilet facilities, could result in poor hygiene practices,” although no such “nonavailability” was actually shown in the survey due to the poor wording of the questions.
• Water Testing: The article stated that irrigation water “is one of the most critical sources of produce contamination,” and raised alarm that 87% of the respondents didn’t test their irrigation water.  But the survey failed to ask about the source of the water.  Many backyard gardeners and small urban farmers use public water supplies that are already tested.  Many also use drip irrigation, which poses a low risk of contamination even with untested water.
• Hydroponics: The article stated that “> 41% of participants could not recognize the difference between hydroponically grown produce and conventional systems.” But the survey asked only whether the respondent felt “that hydroponic produce are safe in comparison to conventional produce.” 41.2% of respondents said they were “not sure.” This does not support a conclusion that the respondents don’t know the difference between “hydroponically grown produce and conventional systems.” Yet the article found that the “variation of responses of growers about organic produce, hydroponic, and conventional grown produce may be due to a lack of knowledge and different perceptions about the food safety issues related to these systems” – turning differences in opinion of the relative safety of hydroponics into a conclusion about growers’ knowledge.

Repeatedly, the survey asked extremely simplistic, incomplete questions from which the authors then drew broad, unsupported conclusions.  This letter reviews only some of the problems with the study’s conclusions.

Many of these issues are immediately apparent to individuals who work with small farms on a regular basis but would not be obvious otherwise.  For example, the alleged connection between manure use and microbial risk would appear obvious to someone without an agricultural background – but the need for more specific data would be equally obvious to someone within agriculture.  In addition, there is significant difference in the knowledge base of those who work with large agricultural entities and those who work with small farms.  For example, large farms almost never use municipal water sources because of the cost, while small farms frequently use these already tested water sources. 

Thus, when the Journal reviews articles on food safety on small farms for possible publication, we would urge you to include at least one individual with expertise specifically in small-scale agriculture as part of the peer review process.

We appreciate the difficult challenge posed in reviewing research in the wide-ranging and complicated field of food safety.  We would be happy to work with you in whatever manner would support the mission of publishing quality research on small farms and food safety.

Sincerely,

Judith McGeary
Executive Director
Farm and Ranch Freedom Alliance