Comment on the Current FMCSA Administration of Carrier Safety Worthiness

By: Paul Stewart, Attorney at Law

August 17, 2011

Please allow me to preface my comments by mentioning my varied experience in the logistics industry. I was for twelve years outside counsel for an intermodal marketing company (IMC), as it grew to be one of the top five IMC’s in the country.

Thereafter, for 17 years I was General Counsel for GST Corporation and NYK Logistics (Americas). I then did consulting for logistics companies, after which I was Chairman and CEO of 4 Elements, Inc., for three years, as it grew from start-up to a $40M logistics company, later to be sold to a much larger company. I have now returned to the private practice of law, focused upon transportation and logistics services, and offer my comments from the perspective of both a transportation lawyer and a business person who has dealt with the real world effects of well meaning, but entirely misdirected and harmful government action.

I mention this experience to give perspective (and hopefully credibility) to my comments on the current manner in which the FMCSA is administering its mandate to improve highway safety as it applies to the trucking industry. More specifically, my comments are directed at the manner in which the FMCSA is administering that portion of its website known as Safety and Fitness Electronic Records System (SAFER) and the Safety Measurement System (SMS).

At one of the worst times in our economic history, the FMCSA is “throwing the baby out with the wash water”, by publishing misleading, unreliable and clearly invalid statistical information to the shipping public. In turn, people who make decisions as to whom will haul their freight are becoming more misled and paranoid every day in their understandable response to what they erroneously trust to be valid information. The unintended effect is already catastrophic for many carriers, and will become more so as reliance upon this invalid information leads to poor business decisions, loss of jobs and decline of the trucking industry.

While there are numerous problems with the SAFER and SMS portions of the FMCSA website, the most fundamental is that the FMCSA is misleading the shipping public into making extreme and erroneous decisions about carrier safety worthiness by posting information that is neither entirely valid, reliable nor necessary. Many new “safety measurement” terms and categories within the SMS are only serving to further confuse the public and are no closer to statistical validity than the SafeStat scores of 20 years ago. Who, among shippers making carrier decisions, really knows what a BASIC calculation is for Unsafe Driving and how the so called “Threshold” is calculated? What statistical validity is there in the varying thresholds, and at what point is this measurement to be utilized by shippers in choosing to discontinue use of a carrier, costing a carrier millions of dollars in revenue, causing lost jobs for drivers and their families, and further capacity shortages for a national economy in real trouble?

For the purpose of these comments, I should clarify my assertion that the statistical information published on SAFER and SMS is both invalid and unreliable by definition within the discipline of statistics. According to accepted statistical methods, a measurement cannot have high validity unless it also has high reliability. Reliability refers generally to measuring a subject consistently over time. It would require a much longer analysis than these comments will permit in order to state all the many reasons that ALL of the current measurement categories of the so-called Behavior Analysis and Safety Improvement Categories (BASICs) are flawed by timeliness of measurements, statistical invalidity for comparison, and highly subjective measurement collection methods. How can these measurements meet any of the statistical reliability requirements when the SMS site notes that many carriers have no recent investigations, reported miles traveled differ greatly from those actually traveled, the FMCSA reports making as many as 800 changes per month in data construction, and even they warn the public that the measurement categories should not be used for making qualitative decisions as to a carrier’s overall safety worthiness? These are all random error possibilities and statistical reliability is inversely related to possibilities for random error.

Validity of a measurement category within the SMS refers to how well-founded that measurement is and how well it corresponds accurately to the real world experience of the carrier. Validity of a measurement tool is considered to be the degree to which the tool measures what it claims to measure and cannot be present where the measurements lack the aforementioned reliability.

How can one who chooses carriers for their transportation needs possibly be better informed to know that a carrier has an “Unsafe Driving” percentile score of 50.8%, and the “Basic Overall Status does not exceed intervention threshold”, when the same site reports “Vehicle Miles Traveled” to be Zero, and “There are no recent investigations”. What does this mean? It is a per se invalid measurement and report for the purposes of current decision making on this carrier’s safety worthiness. As such, it should not be published to the shipping public, any more than unqualified libel or slander should he allowed, when by the powerful innuendo of sanction by the FMCSA such information is used to the economic detriment of that carrier. All of the current measurement categories of the so-called Behavior Analysis and Safety Improvement Categories (BASICs) are similarly flawed by timeliness of incident reporting, statistical invalidity for comparison, and highly subjective measurement collection methods.

I have clients who are losing current and potential business as a result of this confusion, and yet, when we file challenges to the manner in which this data is being used we get answers from FMCSA officials like the following…

”It is unfortunate that someone seeing the violation on your record might assume alcohol use by your driver (when there was none). But, how FMSCA has constructed that system, or how that information is used, is beyond the state’s control. It is not our discretion to ignore a violation because of how that information may later be used by others.” (From ruling on an appeal of data use by FMCSA)

The essence of the problem lies in this example. The FMCSA has responded to outcry and lawsuits from the transportation industry by conceding that the former SafeStat scores were invalid for many reasons, not the least of which was that many carriers’ information was not current. They then created a worse nightmare by producing the SMS suite of measurements, which at best can only be understood by them, even though they admit that some portions of those measurements are not considered valid for their internal use, notwithstanding current public use. However, when jobs and viability of carriers are at stake, they further exacerbate the problem by publication and use of this data, along with state officials, in a manner that wrongly misleads the shipping public. Then, when the unintended effect of this folly is brought to their attention, they give answers like “…But, how FMSCA has constructed that system, or how that information is used, is beyond the state’s control. It is not our discretion to ignore a violation because of how that information may later be used by others.” (Emphasis added)

What should be done about this situation? I appreciate the efforts of certain industry organizations, the compromises that have been achieved, and the leadership of my good friend Hank Seaton in seeking a recognition and rectitude of this folly. However, making slight modifications to the descriptions of certain “data” categories and agreeing to weak disclaimers as to the validity of data, will not slow down the tendency of shippers to rely upon government sanctioned and publicly displayed information in making decisions as to which carriers they should use. We are all still conditioned to believe that our government/administration has all knowing credibility in the information they choose to share with us. In this case, nothing could be further from the truth, and it is in many instances wrongfully costing jobs and economic growth.

Before considering what should be done to rectify this situation, we have to first address a fundamental truism. The FMCSA should be about the business of improving highway safety. No one would offer argument with that purpose. To do so would be like arguing against God and Country. But, in doing so, they should heed the fundamental first premise of physicians, “Do no harm to the patient.”

In an extension of this metaphor, the FMCSA should not allow the least opportunity for any part of their diagnosis to be misleading, and more important, no medication or surgery should be administered to individual patients by laymen who are allowed to see parts of their diagnostic tools, some of which are known to need further refinement to achieve predictive validity. Just as the physician uses many diagnostic tools, the FMCSA should continue to further develop and use the SAFER and SMS, but it should be immediately classified as, “For Internal Use Only”. However, in doing so, they must continue to use a complete analysis in order to render the only information and “prescription” that should be of immediate concern to the shipper. The shipper and general public, in addition to knowing basic information such as license, insurance and operating authority, need only know that the FMCSA has diagnosed any particular carrier as: SATISFACTORY, CONDITIONAL, OR UNSATISFACTORY.

The shipping public should not be empowered by questionable information to participate in sometimes drastic treatment with poor information and/or inferences from information they cannot possibly fully understand. No matter how many disclaimers or descriptions of the limitations of current published data, the opportunity to misconstrue such data should not be made available. The cost is too great in relation to any possible benefit.

Just as using parts of a physicians diagnosis will often lead to mistreatment of the patient, for the FMCSA to continue to publish outdated, unreliable and invalid information, will lead to shippers often applying harsh remedies that may include the business death of carriers and the jobs of their employees. The cure by such inexact interpretation of flawed data is much more devastating than the disease.

Let the FMCSA continue to refine and use all of the statistical measurements they deem important, subject to a fiduciary duty to reach fair and empirical decisions about the safety worthiness of carriers. Such decisions may then be reviewed by careful and more expert analysis and advocacy than allowing laypersons to use only bits and pieces of flawed data put forth in a website forum that at least implies credibility, and at worst provides unreliable, invalid and unnecessary information.