What to Expect in 2015 Audits - Part 2

If you are doing Cybersecurity at a financial institution there are several key takeaways in NCUA Letter No. 15-CU-01, which was released earlier this year. The first major topic discussed in the letter is Cybersecurity, which includes six critical items:

  1. Encrypting sensitive data
  2. Developing a comprehensive information security policy
  3. Performing due diligence over third parties that handle credit union data
  4. Monitoring cybersecurity risk exposure
  5. Monitoring transactions
  6. Testing security measures

In this two-part blog series I will be looking closer at each of these items and outlining FFIEC guidance, describing what we see in the field, and making recommendations based on future expectations.

If you missed Part 1 that is where I covered the first three items on the list. Now on to the second half of the list….

4. Monitoring cybersecurity risk exposure

FFIEC Guidance

Cybersecurity risk management, I must admit, is very special to me. I have been in IT security for 16 years and specifically doing risk assessments for 13 of those years. This is what I do, it’s what I love, and it is the reason I founded Rivial. I wanted to provide a better risk assessment than anybody else and we continue refining the methodology, process, and reports to keep raising the bar.

The FFIEC states that financial institutions must maintain an ongoing information security risk assessment program that effectively

  • Gathers data regarding the information and technology assets of the organization, threats to those assets, vulnerabilities, existing security controls and processes, and the current security standards and requirements;
  • Analyzes the probability and impact associated with the known threats and vulnerabilities to their assets; and
  • Prioritizes the risks present due to threats and vulnerabilities to determine the appropriate level of training, controls, and assurance necessary for effective mitigation.

The key focus by examiners recently, and highlighted in the heading above, is ongoing monitoring. Regulators have been shifting away from annual, point-in-time risk assessments and the NCUA’s letter pushes this approach with renewed gusto.

What we see in the field

Most organizations have some kind of risk assessment in place. It is not a requirement that financial institutions have a third party perform their IT Risk Assessment, like an IT Audit. It is only required that a program be in place.

However, ongoing monitoring of cybersecurity risk is much less likely to be in place. Some institutions we work with—typically over $2-3B in assets—do maintain an ongoing program where risks are identified, measured, and managed over time. Unfortunately most community banks and credit unions don’t have the resources to maintain this level of detail.

One unnecessary and somewhat troubling practice I see occasionally is that some vendors will break out Information Security Risk Assessments and IT Risk Assessments. This approach is not required by any regulation I’ve read, it is counter-productive. The only logical explanations for this practice I can think of are: a basic misunderstanding of Information Risk, or greed on the part of vendors. Why sell one efficient service when you can sell two and charge twice as much.


Make every attempt to simplify the Cybersecurity Risk Assessment process and reporting. When I began performing risk assessments well over a decade ago, I created elaborate structures of threats, vulnerabilities, and residual risks. The hours I poured into the assessments and painstakingly detailed reports were commended, but ultimately the reports sat on a shelf collecting dust. Over the years I simplified, re-organized, simplified, streamlined, and simplified again to create a process that is both efficient and comprehensive. The workload is lower and the results are better.

In some aspects of Cybersecurity, less is more. Don’t need to track all known threat actors and threats to an information system, because the resulting security controls will be the same if you’re measuring risk at a less granular level.
See our ebook guide “How to Get Started with IT Risk Assessments
If you have two separate risk assessments, try to coordinate the two for a more efficient and effective view of Cybersecurity Risk across your organization. If a vendor has provided one or the other, ask why IS and IT weren’t included in the assessment as they should be.

For more advanced organizations that have a formal Enterprise Risk Management program in place, I highly recommend coordinating the Cybersecurity Risk Assessment with the ERM function. Cybersecurity Risk is arguably one of the largest sources of risk for most businesses. It should be accounted for an integrated at the executive level of risk management.

5. Monitoring transactions

FFIEC Guidance

There are several areas in the FFIEC guidance that require transaction monitoring. I wasn’t sure what the impact of this announcement would be to financial institutions, so I put the question to Patrick Truett, Information Security Officer at the NCUA. Mr. Truett provided a detailed response that I think will be very helpful for you:

“NCUA examiners in the near term will still be focusing on the type of monitoring well established by the FFIEC IT Handbook (Retail and Wholesale Payment Systems) and FFIEC Authentication Guidance. NCUA highlights focus areas for e-banking fraud monitoring in its IT exam questionnaires as follows:

Monitoring Items for E-banking

  • Bad login attempts
  • Large bill payments
  • Cross-account transfers
  • Other large or unusual transaction activity
  • Changes to critical information (address/phone number)
  • Administrator activity
  • New enrollees
  • Password resets

Behavior analytics software or fraud detection engines and databases can be great approaches. It is important to note that NCUA does not require credit unions to purchase an automated solution. Some credit unions don’t do any data processing and/or are so small they can do their monitoring manually. However, vendors can do the industry a big service by ensuring their fraud detection solutions are scalable (i.e. affordable) for smaller CUs.

I would also be sure to keep an eye out for new FFIEC standalone guidance issuances that may address this topic. NCUA is highly engaged in developing those. Ultimately, new control practices advocated in those should make it into the FFIEC IT Handbooks.”

Mr. Truett is also seeing an increased emphasis on device reputation. This approach has drawn significant attention from both public and private industry, so it could become a more common component of layered controls to protect e-commerce systems.

What we see in the field

There is definitely a line in the sand based on financial resources. Larger institutions typically have something in place because the volume of transactions warrants the added expense. Smaller institutions, as Mr. Truett stated, are able to perform manual checks on lower volumes of transactions.


Unless vendors begin offering lower cost solutions to community banks and credit unions, there isn’t much we as an industry can do that isn’t already being done. However, as a recommendation I should echo Mr. Truett by keeping an eye out for new FFIEC standalone guidance issuances that may address this topic.

6. Testing security measures

FFIEC Guidance

I think the FFIEC Information Security Handbook covers this area very well. Therefore, I will paste two key sections of the guide without risk of ruining the goodness:

Regarding self-assessments conducted by financial institution staff or the system owner….

“Self-assessments are useful in providing a warning flag to line management so problems can be addressed before they arise in testing reports. Self-assessments may be performed by operations personnel or by vendors under the direction of those at the institution who are responsible for the systems being assessed. Self-assessments may use tools and techniques similar to independently performed audits and penetration tests, and include:

  • Assessing conformance to policies and procedures, including service provider oversight;
  • Scanning for technical vulnerabilities;
  • Verifying that device and network configurations are authorized and changes are properly processed;
  • Verifying that information is stored only where authorized;
  • Reviewing the adequacy of the risk assessment and monitoring plans; and
  • Reviewing test results.”

Regarding independent testing by an audit department or outside vendor….

“Independent tests include penetration tests, audits, and assessments. Independence provides credibility to the test results. To be considered independent, testing personnel should not be responsible for the design, installation, maintenance, and operation of the tested system, or the policies and procedures that guide its operation. The reports generated from the tests should be prepared by individuals who also are independent of the design, installation, maintenance, and operation of the tested system.

Penetration tests, audits, and assessments can use the same set of tools in their methodologies. The nature of the tests, however, is decidedly different. Additionally, the definitions of penetration test and assessment, in particular, are not universally held and have changed over time.

Penetration Tests. A penetration test subjects a system to the real-world attacks selected and conducted by the testing personnel. The benefit of a penetration test is that it identifies the extent to which a system can be compromised before the attack is identified and assesses the response mechanism’s effectiveness. Because a penetration test seldom is a comprehensive test of the system’s security, it should be combined with other monitoring to validate the effectiveness of the security process.

Audits. Auditing compares current practices against a set of standards. Industry groups or institution management may create those standards. Institution management is responsible for demonstrating that the standards it adopts are appropriate for the institution.

Assessments. An assessment is a study to locate security vulnerabilities and identify corrective actions. An assessment differs from an audit by not having a set of standards to test against. It differs from a penetration test by providing the tester with full access to the systems being tested. Assessments may be focused on the security process or the information system. They may also focus on different aspects of the information system, such as one or more hosts or networks.”

What we see in the field

Most financial institutions outsource all or part of their security testing due to the specialized, high-dollar skill set required. This function is generally outsourced at most financial institutions under about $5-10B in assets.

Larger institutions tend to have an Information Security Manager and at least one technical security person. One of these people is likely to possess the knowledge and skills to perform at least rudimentary security testing.

Some financial institutions contract with IT service firms that also provide some type of security testing. Unfortunately in many cases the testing provided by these do-it-all firms is bolted on to existing services, which leads to quality levels that vary widely.


Do as much testing as your staff can do with free or low cost tools, and within their time constraints; outsource the rest to a reputable firm. Before you request proposals from security firms, spend a few minutes educating yourself on the differences between vulnerability assessments and penetration tests. Penetration testing requires more tools and much higher levels of technical skills and knowledge.

For external-facing network assets, do a thorough penetration test annually and after any major changes (e.g. new firewall or IDS device).

I am torn on the topic of Internal Penetration Testing. There is value but only after multiple rounds of vulnerability assessment and process fixes. Some findings are configuration errors and one-time fixes. But the majority of findings such as missing patches, outdated software versions, and default passwords stem from incomplete or inconsistently executed processes. As long as these findings exist, an attacker inside your firewall will be able to gain access to sensitive data. An expensive penetration test isn’t needed to find that out. After several rounds of internal vulnerability assessments and process improvements, it will be time to perform an internal penetration test to fully understand weaknesses.