 |
SEPA HOME |
Report on SEPA Project Checklist Testing (Draft)SummaryThis report provides the results of testing new draft versions of the SEPA environmental checklist for project actions. The Department of Ecology (Ecology) has worked with an advisory committee to investigate changes to the existing SEPA environmental checklist (WAC 197-11-960). The advisory committee previously developed goals or criteria to guide changes to the checklist. Testing took place with the assistance of two volunteer SEPA lead agencies, Thurston County and City of Yakima, and a number of volunteer applicants. The testing simulated a variety of real-world projects. Three versions of the checklist were tested. The results of the testing indicate there is not unanimous agreement about which changes represent an improvement. However, there are elements of all versions that were viewed as an improvement over the existing checklist. At this point in time, Ecology must make a decision about how to proceed with rule making efforts. Ecology must also determine the relationship between any changes to the project checklist and possible changes to the SEPA Rules for nonproject actions. Given the results of testing, Ecology has the following questions for the advisory committee:
Table of ContentsList of Attachments- Click once on attachment in desired format AcknowledgmentsThe Department of Ecology would like to thank the SEPA Checklist Advisory Committee and testing volunteers for their help, without which none of the information in this report would have been possible. We are deeply appreciative of the help provided by our volunteers. Volunteers From Lead Agencies Volunteer Applicants HistoryESHB 1724 was passed by the 1995 legislature and directed the Department of Ecology and the Department of Community, Trade, and Economic Development to revise the State Environmental Policy Act Rules, Chapter 197-11 WAC. Revisions were to reflect the philosophy that project review should start from the fundamental land use planning choices made in the comprehensive plans and development regulations. Project review should not require additional studies or mitigation under SEPA where existing regulations adequately address a proposed project’s probable specific adverse environmental impacts. To meet this objective, the legislature directed "…state agencies to consult with local government and the public to develop a better format than the current environmental checklist to meet this objective" (see notes after RCW 36.70B.030). During the rule revision process, a rule advisory subcommittee evaluated the environmental checklist and developed a revised version that was circulated for review. The comments on this version generally indicated that minor amendments to the current checklist were not sufficient. As a result, the checklist effort was dropped pending completion of other SEPA Rule amendments. After the amendments were completed in 1997, work was again started on the revision of the checklist with the goal of totally amending the form. A Checklist Advisory Committee was formed and began work on the new project checklist. The advisory committee agreed on a checklist purpose (see Attachment A, Environmental/Project Review Checklist Purpose). Purposes of the checklist included, but were not limited to:
As part of the purpose to assist an agency in conducting integrated project review, the checklist was intended to assist in evaluating a proposal’s consistency with comprehensive plans and development regulations and in evaluating compliance with applicable environmental laws and plans. One version of the project checklist was written that integrated GMA and SEPA, and combined the checklist and guidance into one document. In late 1997, comments from some committee members indicated this version was too long and complicated, and that applicants would not be able to get through it. In 1998, another version was drafted that was intended to be simpler and separated the guidance from the checklist. A list of "Suggested Criteria for Evaluating a Draft Checklist" was then developed to guide the process (see Attachment B, Suggested Criteria for Evaluating Draft Checklist ). The criteria included: 1) meeting the stated project checklist purposes (see above), 2) creating a user friendly checklist, and 3) meeting the "list of fundamentals on which the effort to revise the checklist is based". The "list of fundamentals" was identified as:
As part of testing preparation, a Usability Evaluation Plan was drafted for implementing the testing (see Attachment C, Usability Evaluation Plan - word format). The plan identified the following questions to be answered:
The committee asked that both new versions and the existing checklist be tested. In 1999, consultation with a testing expert indicated that a three-way comparison test would be extremely difficult and costly, and beyond our capacity to do. She suggested we try to test one version, so staff created one version (known as the fourth version) that was between the two previously proposed. This version was tested with the use of: 1) one private applicant, 2) two Dept. of Fish and Wildlife applicants, 3) one Dept. of Transportation applicant, and 4) review by a team of Dept. of Natural Resource staff. Feedback from this group was reported to the committee (see Attachment D, Preliminary Results from Testing - word format and Attachment E, Feedback from Testing - word format). Minor changes were made to the form (version 4a) and more testing was planned. The present testing phase began with the testing of Version 4a at Thurston County in December 2000. Six applicants and three lead agency staff used the form and provided feedback. Feedback from the applicants and staff is presented below in the Results section. As a result of Thurston County’s feedback, two other versions of the checklist were developed. There are currently three versions that are part of this testing phase. Overview of Checklist VersionsThe following brief overviews provide a general description of the objectives and format of each checklist. The checklists and guidance documents can be found in attachments F through K (see list of attachments above). Attachment F – Version 4a Draft Environmental Checklist,
II. Testing Description/Scenario BackgroundWith the help of a testing consultant, a testing plan was developed see Attachment C, Usability Evaluation Plan - word format). It was recognized that the test needed to account for a number of variables:
The testing was intended to include representation for each of the variables. It was also designed to test real applicants with their own real projects. Efforts were made to identify applicants and agencies willing to test the new checklist. After extensive efforts were generally ineffective, the test plan was revised and, as a result, may not be as comprehensive as originally envisioned. The final testing was designed to include:
The final testing schedule and locations included:
ProjectsThurston County TestingTwo real projects were selected. One project was a Sports Park on 73.17 acres with a wetland on site. It was described as a multiple-use sports and recreation facility, including eleven lighted softball fields, two soccer fields, a picnic area in an existing oak grove, a mini-golf course, four concession stands, a bowling alley, restaurant, retail and maintenance building and 1000 paved parking spaces. The adjacent areas included a mushroom farm, fire station, residential, vacant land, and the Lacey water tower. Potential issues associated with this project included wetland on site and traffic. The second project was a residential development on 20 acres. It was described as 75 single-family residential lots, 8 townhouse lots and a reserved track for a 42-unit multi-family development. Some of the area would be reserved for landscaping, buffer and recreation. Adjacent areas mostly consisted of open space, one single-family subdivision, and a private park. Potential issues associated with this project included traffic and stormwater. City of YakimaTwo real projects were also selected. The "simple" project was a demolition of a 19,600 square foot building known as "The Armory" in downtown Yakima. There was a long-term plan to build a justice building, although there were no specific plans for this phase of the project. There was some asbestos in the building. A potential issue associated with this project includes waste handling and disposal. The second project was a phased 54-unit condominium with 108 paved parking spaces proposed on 5.61 acres. The site includes a "degraded" creek running along the south border. Adjacent areas include single-family, light industrial, and a racquet club. Potential issues associated with this project include potential impacts to the creek (water quality/quantity and habitat), stormwater, erosion during construction, views and glare, service extensions and road access . The process was basically the same during both tests: Applicants
Lead Agency reviewers
Table A below summarizes the testing at both locations: Table A - Details from Draft Project Checklist Testing
III. ResultsGeneral Impressions
Ecology’s Review of Completed ChecklistReview of specific sectionsSite changes (Cover type changes) in Versions 4a, 4b, and 5 This section regarding site changes generally used the same format and requested the same information in all three versions of the form. It asked applicants to provide before and after acreage or square footage for different cover types (impervious surface, forest, meadow, water surface area, etc.). For all versions, applicants generally filled this section in a very haphazard or incomplete manner. The applicants were probably at a disadvantage due to limitations of the test. This type of information request is present in other states’ environmental review forms (New York and Minnesota). City of Yakima agency reviewer suggested this level of detail was unnecessary and to change it. The change would keep the surface cover types but not ask for specific quantities. Impacts (with minor moderate, and major) in Version 5 Applicants (2) answered this section (Part 4.2) and answered the optional section regarding minor, moderate, and major. Although they did identify some minor impacts, they did not describe any mitigation for these issues or any others in the next section (Part 5). Is there a way to get more thought and information from applicants on mitigation? If there were a link between the two sections, they might be reminded to think about and may provide some relevant information in the mitigation section. Another option would be to put sections 4.1 and 4.2 in guidance and put the mitigation back with the individual sections in Part 2. Special Reports in Versions 4a, 4b, and 5 Version 4a had a Special Reports section in the front, similar to versions 4b and 5. In version 4a, though, applicants were reminded to think about their reports and to provide relevant information from them as they later answered questions about specific issues. In the later sections, the applicants provided more information about Special Reports. One applicant (an experienced consultant) even referred to and provided relevant information from the comprehensive plan. It is possible this may have been due more to the circumstances of the test environment (some applicants felt rushed at the end) and test project than due to the design of the form. (The testing numbers were too small to draw any certain conclusions about this). In version 4b, the reports were listed and referenced later in the document. Applicants did not have a copy of the reports, so it is unknown whether they would have retrieved information from them to insert in Part D, Changes, Impacts and Mitigation. In version 5, the special reports section functioned at a level similar to the existing checklist. The reports were listed and not referenced later. Applicants did not have a copy of the reports, so it is unknown whether they would have retrieved information from them to insert in the changes and mitigation sections of the checklist. Review of different versionsVersion 4a used by Thurston County Based on the way the form was filled out, it appeared there was some confusion about how the screening questions worked and when to move to the next section or when to finish answering the questions for the current section. Generally applicants used the screening questions correctly. Applicants provided more information about possible mitigation. Applicants linked the proposal to relevant reports/plans and reiterated the relevant information (it is possible this may have been a result of the testing conditions rather than the result of the form). Version 4b used by City of Yakima (shortened version of 4a) This version did not retrieve any more information from applicants than version 5. Answers in part C (screening questions) and D (existing, changes, and mitigation) appeared to be less useful than the similar answers in version 5. Applicants did not offer much description of mitigation, although there was more in this version than in version 5. Version 5 used by City of Yakima The sophisticated applicant misunderstood Part 4.1 Existing Conditions and filled out this section as if it referred to project changes. Both applicants left Part 5, Proposed Mitigation, blank. One applicant described some mitigation in Part 4. This applicant was in a hurry at the end and might have spent more time on Part 5 if he had not felt rushed. The form elicited information about existing conditions regarding existing water quality in the creek, current uses of the site, including public use. Because mitigation was at the end and optional, it is possible applicants would forget to say anything. Versions 4b and 5 used by Yakima One key aspect of the more complex proposal was that water from impervious surfaces would go through an oil/water separator before infiltration on site. Neither version appeared to elicit this information from the applicant although they were fully aware of it (they verbally discussed the proposed use of oil/water separators before infiltration) After review of the forms, the city representative noted that it would be helpful to have more questions about stormwater handling because it is such a big issue. QuestionnairesThe applicant and lead agency volunteers answered questionnaires after completing their work on the test checklists. The questionnaires asked the volunteers to provide feedback on attributes of the form with a response using a rating scale (usually from 1 to 5, with 5 being best). The questionnaires also allowed the opportunity to provide narrative responses (included with discussion section of this report). Tables 1b through 6 present the compiled responses to the questionnaires. Responses of volunteers are pooled. However, results are separated to distinguish between test versions and between applicant and lead agency. It should be noted that the sample size for the questionnaire responses is not large enough to strictly rely on the numerical results. However, when combined with feedback from the narrative response and the discussion, the results provide some insight into the effectiveness of the forms. In some cases, the individual response on the questionnaires differs from what was heard during the discussion portion of the testing. Table 1 and Table 2 show responses to a variety of questions addressing the overall usability of the form. Table 1 shows only small differences between the three test versions. Version 4a seemed to be viewed favorably by the applicants, with the only negative response being the difficulty of getting information to complete the form. This is in contrast to the largely negative response from the lead agency on the same version. Version 5 received slightly higher applicant ratings than Version 4b. The lead agency reviewer for Versions 4b and 5 combined his response for both versions. Table 2 shows markedly differing responses by the two agencies on several of initial format related questions, possibly indicating improvements made prior to the City of Yakima test. At the bottom of Table 2, responses from the City of Yakima indicate how well the form integrated information requirements of other forms. Table 1: Applicant Responses on Overall Usability of Form
Table 2: Lead Agency Responses on Overall Usability of Form
Table 3 and Table 4 compare the existing SEPA checklist (WAC 197-11-960) to the versions used in the tests. The Thurston County test showed very different responses by the applicants versus the lead agency; the applicants showed a strong preference for the test version, whereas the lead agency strongly preferred the existing version. The City of Yakima responses were less emphatic, with no clear preference shown by the applicants. There does appear to be some overall agreement that existing checklist is easier and that the test versions are more complete. Table 3: Applicant Responses Comparing Test Version to Existing Checklist
Table 4: Lead Agency Responses Comparing Test Versions to Existing Checklist
Table 5 and Table 6 shows responses to some specific question regarding formatting issues. These questions were added for the City of Yakima test. Table 5 shows that the applicants were generally agreeable on formatting issues, even when the questions were intended to elicit a preference (e.g., two respondents indicated that they liked both lines and white space to write in). The only strongly negative response was on multiple column formatting. The response on questions regarding guidance seems to indicate that the applicants universally like to see guidance in the form itself, although some also like more detail proved in a separate guidance document. Table 6 shows lead agency response to formatting questions that were asked only during the Yakima test. Table 5: Applicant Responses on Specific Formatting Issues(Shows number of responses for each question within category)
Table 6: Lead Agency Responses on Specific Formatting Issues(Shows number of responses for each question within category)
DiscussionVersion 4a - Used by Thurston County Applicant feedback (4 people)
Lead agency feedback (3 staff people)
Versions 4b and 5 - Used in Yakima Applicant feedback (4 people)
Lead agency feedback (1 person)
IV. Conclusions and Questions for Advisory CommitteeThis section provides analysis of how well the project checklist versions performed - based on both test results and Ecology's own review of the checklist versions. A key issue for the revision process, and a range of possible Ecology actions related to the project review form is described. Specific questions Ecology requests input from the SEPA Checklist Advisory Committee is also detailed. Checklist Version ComparisonTable 7 contains a comparison of the checklist versions and the existing checklist against the evaluation criteria and goals that were previously identified. While the test versions were able to improve on the existing checklist in a number of areas, no version was effective in meeting all the criteria. Table 7: Checklist Versions Compared to Evaluation Criteria(How well did checklist version meet criteria? Rated from 1 to 5, with 5 being best)
Key IssueA key issue remains unresolved. Should we proceed with a checklist version that does not support "Starts with decisions already made, relies on previous analysis"? As stated in the Background Section, one of the key objectives of the checklist revision effort was to create a checklist that could be used as a tool to assist agencies to be able to "add" to previous environmental analysis and to fill the "gaps" in environmental regulations. It appears that designing a checklist to accomplish this objective is perceived as making it longer and more difficult. Version 5 and the existing checklist do not accomplish this objective. Changing version 5 to meet this objective would essentially negate many of the strengths of this version. There are many barriers to accomplishing this objective. The checklist is only one tool that, even if perfectly designed, could not alone result in full success in meeting this objective. Practically speaking, the objective is a difficult one. With the current tracking system for existing environmental data and the challenges of accessing relevant information in plans, rules, and ordinances, it is difficult for any one person (applicant or agency staff) to identify: 1) analysis that is already done and 2) all local, state, and federal regulations that fully or partly mitigate impacts. The first checklist version, known as version 2 (See Attachment O, Version 2 Draft Environmental Checklist) was probably formatted in a manner that would best accomplish this checklist objective. It was not well received by several committee members. It was felt that average applicants could not fill it out. Guidance was integrated into the checklist, resulting in a lot of text and discussion for each question. Terminology was fairly technical. As a result of various factors, this version did not get tested and was modified and integrated with version 3. Because this version did not get tested, it is unknown how well it would have accomplished the objective. Direction and Next StepsEcology would like to move forward to resolution of the project checklist issue, regardless of whether we drop the effort or move forward with broader public review of a proposed form. If we are going to make rule changes to the project checklist, it would best occur simultaneous with nonproject revisions. At a minimum, Ecology would like to proceed with creating some guidance materials for checklist users and also making some minor changes to the existing checklist. From the testing efforts, we received positive feedback on guidance; ideally the guidance would be included both in the form itself and in a more detailed companion guidance document. Additionally, we would like to further investigate adding a requirement for inclusion of a site map with the checklist. If the SEPA Checklist Advisory can reach consensus on a version of the checklist to pursue, further improvement can be made to that version and Ecology can proceed with a broader public review. If there is no consensus among advisory committee members, Ecology will likely focus efforts on guidance and minor changes to the existing checklist (e.g., include GMA terminology). Questions for SEPA Checklist Advisory CommitteeAt a future advisory committee meeting, Ecology would like to hear responses from the committee members to the following questions. For members not attending, we would like to receive response by the end of April, 2001.
For general questions or comments concerning SEPA, please e-mail the SEPA Unit or call (360) 407-6922. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © Washington State Department of Ecology. See http://www.ecy.wa.gov/copyright.html.
|
|
