Relevant Technologies: Secret Service Cyber Forensics Team Available to Assist Corporate America

< Back to Article List

Should we believe the Federal Computer Security Report Card?
By Laura Taylor
January 12, 2009

First Some Background Information
The Federal Computer Security Report Card is published each year by the House Oversight and Government Reform Committee to show how well U.S. federal agencies complied with the Federal Information Security Management Act of 2002 (FISMA) in the prior year. The determination of grades is made by the U.S. Government Reform Committee after inspecting reports put together by OMB, agency inspectors, and the GAO. The OMB FISMA Report to Congress does not assign grades – it identifies trends, publishes percentages of compliance, and offers qualitative (Excellent, Good, Satisfactory, Poor etc) assessments of 25 large agencies.

Of additional interest is that the OMB FISMA Report to Congress also provides the criteria used for agencies to obtain a Red, Yellow, or Green on the E-Government Scorecard. The E-Government Report Card is one of five categories in the Executive Branch Management Scorecard. In 2007, the criteria to obtain a green rating on the E-Government Scorecard were:

Inspector General or Agency Head verification of effectiveness of the Department-wide IT security remediation process
Inspector General or Agency Head rating of agency C&A process as “Satisfactory” or better
The agency had 90 percent of all IT systems properly secured (Certified and Accredited).

As of July 1, 2008, the criteria to obtain a green rating were changed to:

All systems certified and accredited
Systems installed and maintained in accordance with security configurations
A Privacy Impact Assessment (PIA) has been conducted and is publicly posted for 90% of applicable systems
A system of records notice (SORN) has been published for 90% of all systems with Personally Identifiable Information (PII)
Has an agreed-upon plan to meet communication requirements for a COOP/COG (Continuity of Operations/Continuity of Government).

For 2009, OMB has formally incorporated three performance measures into their OMB 300 exhibit audits that are related to FISMA:

Inspector General assessment of the quality of the agency’s C&A process
Inspector General reviews effectiveness of the POA&M process
Inspector General assessment of the quality of the PIA process.

My reason for pointing out the above bullet points is because the OMB 300 audit could affect how much funding an agency receives from year to year for FISMA compliance. A poor rating on an OMB 300 audit could adversely affect funding to comply with FISMA, which could then make it much harder to actually fund the cumbersome compliance process.

Since the passing of FISMA, much due diligence and budgetary expenditures have been allocated to information security programs. Though new funding for resources have clearly improved the security posture of U.S. federal information systems, there are some grave misconceptions about the well-publicized Federal Computer Security Report Card.

Misconceptions and Fallacies
Each year, the Federal Computer Security Report Card comes out in the spring with grades published for the prior year. For example, the 2008 report card will come out in April 2009.

The report card is a bit misleading to the general public and in fact can cause unnecessary public alarm. When the general public reads that a U.S. federal agency has received an “F” on the Federal Computer Security Report Card, it gives the impression that our government owned information systems and networks are wildly insecure. While it is certainly possible that an agency could receive an “F” on the report card for having wildly insecure systems, they could also receive an F not because their systems and networks are insecure, but because the agency did not prove that they were secure. It is not enough to actually have secure networks and systems – to get an A you have to prove you have secure networks and systems. Proving your systems and networks are secure is extremely resource intensive, and requires lots of testing, lots of documentation and paperwork, and then verification by independent auditors. A good analogy is that a person can actually be in great shape and good health, even if they have not had any tests, or a doctor’s verification to prove that they are in great shape and good health. The compliance process of proving that your systems and networks are secure is known as Certification and Accreditation (C&A).

An F can mean a lot of different things, and not including poor technical security controls, could mean any combination of the following problems:

Security configurations are not well documented
Not all systems have undergone C&A
Contingency Plans have not been tested or documented
Incident Response Plans have not been tested or documented
Security controls have been implemented but not tested or documented
Security controls have been documented, but not tested or verified

The above list is not all-inclusive. There are many more possibilities than I can name here. The take away point is that an F may not be what it appears to be and that is a problem for anyone trying to read and understand the Federal Computer Security Report Card.

Compliance Anomalies Yield Mixed Results
An agency could have exceptional security in place, but if the security mechanisms, controls, policies, and procedures are not well documented, or incorrectly documented, there is a good chance the agency could receive an F. Keeping that in mind, an agency that receives an F could possibly even have better security than an agency that receives a C or a B. If you have mediocre security in place, but you document the security controls, policies, procedures, and contingency plans at least well enough to receive a passing accreditation that a Designated Approving Authority will sign up to, it is altogether conceivable that you could receive a better grade than an agency that has nothing documented, but has sound technical security controls in place.

There is certainly something to be said for documenting how things work, as far as security goes. And there are many reasons why documenting security controls, policies, standards, and procedures are a good idea. However, lack of documentation and a poor grade gives the impression that no security exists at all. While it is possible for zero security to be the case, it’s important to understand that if no security exists at all, and it’s not documented, a poor grade will still be issued. What needs to be resolved is how to differentiate between systems and networks that have no security at all in place with no documentation, and agencies that actually do have security controls in place but still have not enough documentation to be fully accredited for an Authority to Operate.

With all the discussed considerations, should you choose to believe the Federal Computer Security Report Card, do take it with a grain of salt.