Often these "agreed upon" standards are set by one party and accepted by another who is usually forced to trust the authority of regulators, along with thier own personal judgement. For example, in a democracy, a political party (faction) will be elected to pass laws to set a safety standard.
Some claim safety can be objectively and globally defined by referring to "ecoregions of origin" and the "native or not" status of organisms - there must be a rigorous audit of organisms crossing borders due to human action. Safe trade is cross-border trade which satisfies this definition. It is promoted by Greenpeace, biosafety and biosecurity activists.
The concept of what is fair seems always to depend on what participants and observers individually feel is safe and done. There is a range of audit procedures one might apply to determine what is "not safe". Factions make it impossible to settle on one strict and global definition, so this is a contested term - see glossary for other such contested terms.