When an organization undergoes an audit, there’s often a large amount of internal controls that an auditor has to review. However, to make this process more efficient, auditors are likely to use sampling whenever the population being tested is uniform and there’s standards that are applied across the board.
How Do Auditors Use Sampling?
At KirkpatrickPrice, our auditors will sample a size of anywhere from 10 to 30 percent of any given population. This also means that the least number we will ever take of a population is three. So, for example, let’s say that an organization hires three employees that year, all of which read and signed acknowledgements that they understood their employee handbook. To verify that this is true, an auditor would test the entire population of three new employees. Likewise, if 100 new employees were hired that year, an auditor might only evaluate ten employees, or ten percent of the population, to ensure that this took place.
It’s important to note that when an auditor uses sampling during the assessment process, it’s randomly selected. This helps the auditor provide a fair, thorough, and accurate opinion on whether or not the controls are in place and operating effectively.
When an auditor performs tests of controls, it may be appropriate to apply sampling. Sampling is used whenever the population of something is uniform and there’s standards that are applied across the board. For example, if we have a large population of, let’s say, 1,000. Our auditor may only take a maximum of 30 out of that total population to perform their testing. We like to use the percentage of 10 percent with a maximum of 30 percent. So, if you had 100 in a population, then we would take 10 percent of that and test 10 of the items. The least number we will ever take of a population is three. For example, if you only have three new employees who are hired and they had employee handbook acknowledgements that were signed, we would take all three of those. On the other hand, if 100 people were hired in the last year, we would take 10 percent of that, which would be 10 from that population. Whenever an auditor is applying sampling, the auditor will look at the total population, and will look at the documentation that’s produced from that population and randomly select the evidence from that in order to determine whether or not they can be reasonably assured that the control has been operating effectively.
[/av_toggle]
[/av_toggle_container]
How Do Auditors Perform Tests of Controls?
In order for an audit firm to be able to provide reasonable assurance and issue an opinion on an organization’s compliance with SOC 1 or SOC 2 audits, they have to test the internal controls that each organization has in place and verify that they are working as intended. To do this, auditors typically perform three types of tests of controls: interviews, reviews, and observations.
Interview: Interviews play a critical role in an assessment because auditors are able to talk to an organization’s employees – the people responsible for effectively implementing your internal controls. During the interview, auditors will want to find that an organization’s employees have an understanding of the purpose of the controls they’re responsible for and how they have been trained to effectively implement them.
Review: During an audit, auditors need to ensure that organizations are doing what they say they’re going to do, and to verify that this is happening, they’ll want to review documentation, such as policies and procedures. For example, if an organization’s policies and procedures say that when they hire employees, they are put through initial security awareness training and then are to take courses annually thereafter, an auditor will want to see documentation, such as completion reports, to ensure this is taking place.
Observation: While interviewing and physically reviewing documents allow auditors to test an organization’s internal controls, observing how those controls are implemented is also a way auditors can verify that controls are implemented and functioning as intended. For example, if your organization claims that you use antivirus software that updates every day, every four hours, an auditor would want to observe that that is taking place.
To find out how your auditor completed these tests of controls, organizations can refer to the section in their audit report labeled “Auditor’s Test of Controls.” This is where audit firms disclose what they did to test an organization’s controls and how they based their opinion upon those tests.
In your audit report, you’ll see a section that’s titled “Auditor’s Test of Controls.” This is the section where we disclose what we did and what we used to base our opinion upon. We’re trying to achieve that level of reasonable assurance and our tests help us to get there. There are three types of tests that we typically perform here at KirkpatrickPrice: review, observation, and interview. Interview is where we talk to your employees, the people who are responsible for your controls. We make sure that they understand the purpose of the control and what it is that they’ve been trained to do to execute their tasks at their job. Review is usually reviewing documentation. If you state that you have a policy that governs your information security practices or you have a policy that governs your hiring and termination practices, you have training materials that your employees follow after they’re hired and annually thereafter, we will review evidence of that documentation and those policies to make sure that those things are in place, enforced, and updated on an on-going basis. Finally, we have observation as a test of control. This is where you might say to us, “We have put this system in place to monitor the health of our network. We have this software development lifecycle that our developers follow. We have antivirus installed and it updates every day every four hours.” These are things that we will observe in order to make sure the controls are actually there, in place, and operating effectively. Anytime that we perform these tests and we find something that’s not working the way it’s supposed to, we bring those issues to management and let you know about those things immediately, so that you can remediate anything that’s critical in nature.
So, how are those risks different? How to they affect an auditor while performing SOC 1 or SOC 2 audits? Let’s discuss.
What is Audit Risk?
According to the AICPA, audit risk is “the risk that the auditor expresses an inappropriate audit opinion when financial statements are materially misstated. Audit risk is a function of the risks of material misstatement and detection risk.”
Essentially, audit risk includes the risk that an auditor did not perform their due diligence when assessing an organization’s compliance with the SOC 1 or SOC 2 frameworks, which might include failing to test something, missing a critical piece of evidence, or something else in the audit was incorrect. Audit risk ultimately refers to the risk that an CPA firm issues an inaccurate opinion of an organization’s internal controls.
What is Control Risk?
During SOC 1 and SOC 2 audits, control risks represent the chances that your controls are not operating effectively or that the failure of a control could lead to material misstatement in financial statements. Control risk takes into account the potential of error from both humans and automated processes. Why? Because humans are inherently inclined to make mistakes, and no automated process is completely error-free.
Although there is always some level of risk, throughout the assessment process, an auditor will work to mitigate control risks as much as possible by designing tests to obtain reasonable assurance that the controls are operating effectively and that their audit opinion is going to be accurate and based on good results.
What is Detection Risk?
In order for auditing to be effective, an auditor must be able to detect misstatements throughout the assessment. Considering this, detection risk is the risk that an auditor will fail to detect something that’s in existence. An auditor can reduce the level of detection risk by designing tests of policies and procedures and applying sampling to help give reasonable assurance that a control is in place and operating effectively.
The Importance of Proper Risk Management & SOC Audits
Each of these risk types must be accounted for in a risk management program that identifies possible threats, assesses existing controls, and documents potential risks so that an organization’s policies and procedures can address them.
High-level risk management best practices are similar for all risk types, but clients need to understand the risks auditors are considering, how they design tests to improve risk detection, and how they work to control and mitigate potential sources of risk.
One of the things that I really believe is important for our clients to understand is the type of risk that our auditor is thinking about as they’re working with you on your audit engagement. We think about audit risk, control risk, and detection risk. Audit risk is the chance that something in our audit is wrong, we missed something, or we didn’t test something. In other words, our opinion that we issued is incorrect because there was something that we should have found. Obviously, we want that risk to be as low as possible, and we’re always thinking about that as we do our work. Control risk is the chance that the control we’re testing is not operating the way it’s supposed to operate. For example, controls fail and if you have a person who is responsible for monitoring a system, people fail and make mistakes. There are inherent limitations to humans doing something, so there is always a chance of a control not operating effectively. What about technology? Technology has failures and anomalies. Sometimes it’s down or it’s not able to connect or do what it’s supposed to do, so that control can fail. That’s control risk: what is the chance that this particular control won’t operate in the way that it was intended to operate? In order for us to address those levels of risk, we as auditors design tests in order to sample a good amount of systems to obtain reasonable assurance that these controls are operating effectively and that our audit opinion is going to be accurate and based on good results. We will perform more tests the higher the level of risk that the control might fail and less tests depending on the lower level of risk that the control might fail. Ultimately, it’s all about performing the audit correctly according to professional standards, because it is an opinion and validation of your controls that your clients rely upon. They rely upon your auditor to do a quality job, and you should expect and demand that as well to make sure your environment is tested as stringently as can be, so that nothing is missed, and nothing is left undone before we issue an opinion.
[/av_toggle]
[/av_toggle_container]
During the initial scoping phases of an organization’s audit engagement, your auditor will partner with you to help you narrow down the third-party vendors to be included in your engagement. In order to ensure that your organization’s security posture is and remains strong, you need to consider the impact that the third-party vendors you’ve entrusted sensitive data with could have on your organization. This means that you’ll need to be able to list who your third-party vendors are, what services they provide to you, and whether they’ve gone through audits themselves. Knowing this information will help you determine whether or not you need to carve them out of your audit or include them. What’s the difference between carving out or including third-party vendors in an audit? Let’s take a look.
Carve-Out vs. Inclusive Method: What’s the Difference
When an organization opts to use the inclusive method for their third-party vendors, this means that they will be included in the scope of the audit. This also implies that the third-party has not had an audit of their controls performed, and the organization being audited wants to make sure that the third-party vendors they’ve partnered with are doing what they say they’re doing to protect their sensitive assets. When using the inclusive method, auditors will perform a site visit, test personnel, interview them, and collect evidence on their controls. On the other hand, when an organization opts to carve-out their third-party vendors, this means that they will not be included in the audit and your audit firm will not issue an opinion on any controls that they have in place that you rely upon to deliver your services. Typically, this implies that the third-party vendor has their own audit report to provide to your audit firm for review and no further action is required on their behalf.
Need help determining if you should carve-out or include your third-party vendors in your audit? Contact us today.
One of the decisions that needs to be made for your audit is how to treat your third-party service providers. There are two methods – carve-out and inclusive – and I’m going to explain the difference between the two. If you carve-out your third-party service provider, that means that we do not issue an opinion on any controls that they have in place that you rely upon to deliver your services. When you hand your report to your client, they are very likely going to ask how they can validate the controls of that third-party service provider. They often want to know if they have an audit report that they’ve had performed so they can review it. If they haven’t had an audit report, they might question how they can be sure if that third-party you’re partnered with is doing what they say they’re doing to protect their data. The inclusive method is where we include the third-party service provider in your audit. We visit them, test them, interview them, collect evidence from them – just like we do for the service organization. In the report then, it would talk about the controls that were in place at not only the service organization but the sub-service organization, or third-party service provider, as well. So, think about which third parties you work with and whether or not they have their own audit report, and whether or not they should be included or carved-out as part of your audit process.
[/av_toggle]
[/av_toggle_container]
Do You Need a Gap Analysis?
If it’s your first time pursuing compliance for any framework – whether it’s SOC 1, SOC 2, PCI DSS, HIPAA, GDPR, etc. – we strongly recommend beginning your engagement with a gap analysis. At KirkpatrickPrice, we’re committed to helping our clients get the most out of their audit, which means that we don’t want you to fail due to lack of preparation. That’s why our gap analysis service is specifically designed to help you prepare for the audit so that you can meet your compliance goals. How does the gap analysis process work? Organizations will be partnered with an Information Security Specialists and an Audit Support Professional to identify any operational, reporting, and compliance gaps and will then offer advice on strategies for remediation. Ultimately, gap analyses ask and answer, “How are we doing compared to what regulations require?”
Do You Need a Remote or Onsite Gap Analysis?
Many of our clients ask us whether or not they should do a remote or onsite gap analysis, and the answer really boils down to how prepared you want to be. Many organizations believe that remote gap analyses are the most convenient option — organizations simply have to upload documentation and evidence into our Online Audit Manager for review and attend conference calls with one of our Information Security Specialists over a two- to three-week period. For organizations who opt to do an onsite gap analysis, it typically is a much more intensive experience. An auditor will come on site over a three- to five-day period to review documentation and evidence and interview personnel. Regardless, whether an organization decides to undergo a remote or onsite gap analysis, they’ll leave with a better understanding of how to remedy vulnerabilities found, a timeline and strategies for doing so, and resources to guide them along their remediation journey.
If it’s your first time going through an audit of a specific framework, let us be your guide. Contact us today for more information on the value of gap analysis and what KirkpatrickPrice’s process is.
We commonly receive inquiries about how to get started with an audit. People are worried that they aren’t ready for the audit, and the question is always along the lines of “What can we do to prepare? What are the ‘gotcha’ areas that we need to be concerned with?” One of the ways that we love to help our clients with this is with a service called a gap analysis. One of our senior, expert-level auditors will be assigned to you and will perform either a remote or in-person gap analysis. We walk through the requirements of the audit, and we help you identify any gaps in your policies, your procedures, your controls, or anything you need to do to quickly address any gaps you have in compliance for the particular audit framework that you’re seeking to comply with. We can perform a gap analysis anywhere in the world. We travel overseas and we perform things remotely in a virtual manner in order to help you understand what you need to do as quickly as possible and get you on the road to completing your audit.
