Description

Control Systems Engineer III

Overview
This is the third level position in the Control Systems Engineering Group responsible for developing and implementing supervisory control and data acquisition (SCADA), energy management (EMS), or other systems typically used by system operators to monitor and control integrated utility systems. Designs networked system interfaces and applications used to supervise and operate remote terminal units (RTUs), programmable logic controllers (PLCs), or similar intelligent electronic devices (IED) in real-time. Implements cyber security controls that conform to North American Electric Reliability Corporation (NERC) standards. Documents system processes and controls. A fully skilled employee in this role is able to perform all of the essential functions listed below, and will work primarily on moderate-high risk and moderately-highly complex work assignments. Leads moderately complex projects.

Responsibilities

• SCADA (EMS/GMS): Perform work on advanced networked systems with a moderate-high risk level to operational compliance, revenue stream, equipment and human safety. Design, develop, implement, document and maintain Transmission, Distribution, Generation, Hydro Optimization, Hatcheries, Water/Wastewater, Mid-Columbia Hourly Coordination, Security and Compliance systems and related human machine interfaces, database structures, control models, multi-system architecture, communication topology, code, data storage, tools and procedures. Follow change management procedures. Design work practices. Provide control room Operators, Energy Planning & Trading, Outage Coordinators, Engineers and Project teams with SCADA system tools, prototypes, processes and procedures; data analysis, visualization, and analytical tools; system documentation; and engineering analysis and technical support. Develop integrations and interoperability for multiple systems. Identify system issues and develop solutions. Purchase, implement, document and maintain servers, workstations, and network equipment. Configure and manage multiple operating systems. Develop new or existing vendor applications. Develop test scripts and conduct testing. Investigate, evaluate and benchmark new display and tool technologies. Act as a subject matter expert for multiple system performance standards, and contributes to the development strategy for tools and applications. Mentor and train less experienced engineers. Maintain critical asset information. Maintain knowledge of emerging trends and industry best practices. Represent the District in industry meetings, discussions and forums.
• Reliability Compliance: Perform work on advanced networked systems with a moderate-high level of risk to regulatory compliance and company reputation. Create and maintain program documentation and perform applicable processes to meet NERC CIP requirements. Provide support for internal and external audits, including annual self-certifications, and peer review, audit worksheet maintenance, and audit interviews. Evaluate NERC alerts. Develop and implement mitigation actions to resolve potential compliance concerns. Create work orders for other work groups to address NERC CIP-related requirements. Act as a Standard Owner, Subject Matter Expert or Contributor to NERC CIP Standards. Research regulations by reviewing regulatory bulletins and other sources of information. Prepare reports by collecting, analyzing and summarizing information. Develop, implement, document and maintain technical tools and security controls required to meet compliance objectives identified in the program documentation. Attend compliance working groups, forums and related industry events.
• Cyber Security: Perform work on advanced networked systems with moderate-high level of risk to operations, regulatory compliance and company reputation. Define, develop, and document secure architecture and solutions. Implement technical security solution policies in alignment with business risk. Prepare and maintain technical user guides, System Operating Procedures, security architecture documentation and diagrams. Provide recommendations and strategies for ensuring remediation of gaps and deficiencies as projects develop. Monitor alerts and logs. Audit security controls. Perform security risk assessments and develop mitigation plans. Develop and maintain security tools and technologies such as event management; multi-factor authentication; intrusion detection; logging, alerting and monitoring; data retention and restoration; anti-virus/malware detection; software and security updates; configuration management; asset management and baseline monitoring; system hardening; network access control; least access privilege implementation; vulnerability assessments. Conduct technical reviews. Conduct cyber security incident response plan tests. Coordinate third-party penetration tests. Evaluate and recommend new and emerging security products and technologies.
• Project Management: Perform work on advanced projects with moderate-high risk to operations and revenue stream. Lead multi-disciplinary project teams that may include consultants, contractors, legal, procurement, and other District departments. Work with District Project Managers to provide tasks and time estimates for projects requiring CSE time. Provide technical support to project teams assigned to designing, implementing or modifying District control systems. Perform project intake processes to add project work to DWP. Organize technical meetings associated with project work that impacts SCADA/EMS design.
• Maintain regular and predictable attendance: Perform related duties and responsibilities as required. Comply with District policies. Complete all required training. Maintain a working knowledge and comply with District safety procedures and specific safety requirements of this position, and those in accordance with applicable provisions of the Revised Code of Washington (RCW) and Washington Administrative Code (WAC).
• As an essential function, drive a motorized vehicle while conducting business on behalf of the District.

Reports To
Manager - Control Systems Engineering

Overtime exemption
Exempt

Control Systems Engineer IV

 Overview
This is the highest level individual contributor (non-leadership) position in the Control Systems Engineering Group responsible for developing and implementing supervisory control and data acquisition (SCADA), energy management (EMS), or other systems typically used by system operators to monitor and control multiple integrated utility systems. Designs networked system interfaces and applications used to supervise and operate remote terminal units (RTUs), programmable logic controllers (PLCs), or similar intelligent electronic devices (IED) in real-time. Implements cyber security controls that conform with North American Electric Reliability Corporation (NERC) standards. Documents system processes and controls. Leads complex projects. A fully skilled employee in this role is able to perform all of the essential functions listed below, and will primarily work on the highest risk and most complex work assignments.

Responsibilities

• SCADA (EMS/GMS): Perform work on highly complex, interconnected, interdependent systems with the highest levels of risk to operational compliance, revenue stream, equipment and human safety. Design, develop, implement, document and maintain Transmission, Distribution, Generation, Hydro Optimization, Hatcheries, Water/Wastewater, Mid-Columbia Hourly Coordination, Security and Compliance systems and related human machine interfaces, database structures, control system models, multi-system architecture, communication topology, advanced code, data storage, tools and procedures. Follow change management procedures. Design work practices. Provide control room Operators, Energy Planning & Trading, Outage Coordinators, Engineers and Project teams with SCADA system tools, prototypes, processes and procedures; data analysis, visualization, and analytical tools; system documentation; and engineering analysis and technical support. Conducts impact assessment studies and develops mitigation plans. Develop integrations and interoperability for multiple systems. Identify system issues and develop solutions. Purchase, implement, document and maintain servers, workstations, and network equipment. Configure and manage multiple operating systems. Develop new or existing vendor applications to full potential. Develop test scripts and conduct testing. Investigate, evaluate and benchmark new display and tool technologies. Act as a subject matter expert for multiple system performance standards, and contributes to the development strategy for tools and applications. Mentor and train less experienced engineers. Maintain critical asset information. Maintain knowledge of emerging trends and industry best practices. Represent the District in industry meetings, discussions and forums.
• Reliability Compliance: Perform work on highly complex, interconnected, and interdependent systems with the highest level of risk to regulatory compliance and company reputation. Create and maintain program documentation and perform applicable processes to meet NERC CIP requirements. Provide support for internal and external audits, including annual self-certifications, and peer review, audit worksheet maintenance, and audit interviews. Evaluate and respond to NERC alerts. Design, develop and implement mitigation actions to resolve potential compliance concerns. Create work orders for other work groups to address NERC CIP-related requirements. Act as a Standard Owner, Subject Matter Expert or Contributor to NERC CIP Standards. Research regulations by reviewing regulatory bulletins and other sources of information. Prepare reports by collecting, analyzing and summarizing information. Design, develop, implement, document and maintain technical tools and security controls required to meet compliance objectives identified in the program documentation. Represent the District at compliance working groups, forums and related industry events.
• Cyber Security: Perform work on highly complex, interconnected, interdependent systems with the highest level of risk to operations, regulatory compliance and company reputation. Define, develop and document secure architecture and solutions. Architect and implement technical security solution policies in alignment with business risk. Design, prepare and maintain technical user guides, System Operating Procedures, security architecture documentation and diagrams. Provide recommendations and strategies for ensuring remediation of gaps and deficiencies as projects develop. Monitor alerts and logs. Audit security controls. Perform security risk assessments and develop mitigation plans. Design, develop and maintain security tools and technologies such as event management; multi-factor authentication; intrusion detection; logging, alerting and monitoring; data retention and restoration; anti-virus/malware detection; software and security updates; configuration management; asset management and baseline monitoring; system hardening; network access control; least access privilege implementation; vulnerability assessments. Analyze requirements and recommend additional safeguards. Conduct technical reviews. Conduct cyber security incident response plan tests. Coordinate third-party penetration tests. Define and implement IPS/IDS network standards, authentication and controls. Evaluate and recommend new and emerging security products and technologies.
• Project Management: Perform work on highly complex, multi-system projects with the highest level of risk to operations and revenue stream. Perform project management activities including schedule, budget and execution for complex projects. Lead multi-disciplinary and multi-functional teams that may include consultants, contractors, legal, procurement, and other District departments. Work with Contractors, Vendors, District Project Managers to provide tasks and time estimates for projects requiring CSE time. Provide technical support to project teams assigned to designing, implementing or modifying District control systems. Perform project intake processes to add project work to DWP. Organize technical meetings associated with project work that impacts SCADA/EMS design.
• Maintain regular and predictable attendance: Perform related duties and responsibilities as required. Comply with District policies. Complete all required training. Maintain a working knowledge and comply with District safety procedures and specific safety requirements of this position, and those in accordance with applicable provisions of the Revised Code of Washington (RCW) and Washington Administrative Code (WAC).
• As an essential function, drive a motorized vehicle while conducting business on behalf of the District.

Reports To
Manager - Control Systems Engineering

Overtime exemption
Exempt

Skills and Abilities
A fully skilled incumbent will have the following knowledge skills & abilities:

• Industrial control systems design, selection and specification of equipment and design of industrial communication and control networks for systems including Transmission, Distribution, Generation, Hydro Optimization, Hatcheries, Water/Wastewater, Mid-Columbia Hourly Coordination, Security and Compliance
• Engineering and Industrial Control Systems security principles and practices
• Distributed control (DCS), host centric (SCADA) and Embedded System environments utilizing three (3) or more computing platforms, PLCs and RTUs
• Control systems theory and its practical application to process control and field instrumentation
• Computer programming in four (4) or more languages, preferably a combination of object-oriented and script languages
• Control protocols including DNP, OPC, ICCP, Allen-Bradley, L&G 8979 and MODBUS
• Data communications protocols, networking equipment and computer operating systems
• Operating system hardening, configuration, troubleshooting, maintenance, and optimization techniques
• Automation, system and communications integration methodologies
• River control and hourly coordination
• Power generation for hydro unit operation
• Energy accounting
• Regulatory laws, standards and requirements including FERC/ NERC Reliability Standards
• Complex spreadsheets and database applications
• Project management concepts, principles and practices
• Analyze threat information from multiple sources, disciplines, and agencies across the Intelligence Community.
• Vulnerability Assessment and Management - Conduct assessments of threats and vulnerabilities. Determine deviations from acceptable configurations, enterprise or local policy. Assess the level of risk, and develop and/or recommend appropriate mitigation countermeasures in operational and non-operational situations.
• Use defensive measures and data from a variety of sources to identify, analyze, and report events that occur or might occur within the network in order to protect information, information systems, and networks from threats.
• Synthesize and place intelligence information in context; draws insights about the possible implications.
• Collect, process, preserve, analyze, and present computer-related evidence in support of network vulnerability mitigation, counterintelligence or law enforcement investigations.
• Work with a field support team to diagnose issues and propose resolution
• Respond to crisis or urgent situations within the pertinent domain to mitigate immediate and potential threats. Use mitigation, preparedness, and response and recovery approaches, as needed to maximize survival of life, preservation of property, and information security. Investigates and analyzes all relevant response activities.
• Participates as a reviewer in technical reviews
• Strong reasoning, analytical and problem solving skills
• Data extraction, data analysis, research and report development
• Understand, interpret and contribute to the development of contract documents
• Communicate logically and clearly, both orally and in writing
• Interpret engineering problems and solutions and communicate technical information effectively to non-department and non-technical staff
• Technical writing
• Estimates project cost and schedule
• Aware of the "business" of control engineering as well as the technical design issues
• Seek long term benefits to the overall project when determining system and control design approach
• Actively look ahead to plan for and respond to project challenges
• Reviews company policies and industry practices, and develops and implements procedures, and standards.
• Learn new computer systems from documentation and training
• Act as a Subject Matter Expert for 4 or more SCADA, compliance and associated supporting systems
• Diagnose problems to isolate cause
• Maintain, troubleshoot and modify program HMI components
• Analyze programming, observe monitoring signals, troubleshoot systems
• Think beyond past approaches to propose and implement practical approaches to control design
• Identify solutions of high complexity by analyzing data, evaluating alternatives and recommending an appropriate course of action
• Apply technical knowledge of legacy systems to design new functionality
• Write clear system requirements and test plans for Security Controls and Automation
• Apply the principles of software management to organize and maintain software integrity
• Apply mathematical concepts such as scaling, conversions, raising to a power, logic ops (AND/OR/XOR), integration, totalization, linear fit approximation and extrapolation
• Design and apply open-loop, closed-loop, time-weighted, and weighted-sum control theory algorithms for linear and non-linear systems
• Provide technical direction and act as a Subject Matter Expert as it relates to cyber security in industrial control systems
• Carry out highly complex technical tasks
• Identify, research and gather relevant information from a variety of sources
• Meet deadlines on multiple simultaneous priorities for multiple stakeholders
• Respond to after-hours call-outs
• Represent District at industry meetings and conferences, which may involve travel
• Train other engineers and dedicated District training professional staff
• Clearly communicate technical information to end users and stakeholders
• To perform this job successfully, an individual is required to have advanced/expert knowledge of Microsoft Excel; Microsoft Word; Monarch; Intellution; PC Hardware; Servers; General Office Equipment; PI Historian software; Compliance tools; Microsoft Windows and Linux server and workstation operating systems; Network equipment operating systems; Microsoft Active Directory; SQL Server; Visual Studio; Enterprise security management software;
• Telecommunication software; PLC, RTU, Meter, and IED hardware; C-family and scripting (PowerShell and Unix Shell) programming languages; and intermediate knowledge of Microsoft Outlook; Primavera; Maximo Asset Management; and Visio.
• Must be proficient in reading, writing and speaking English.

Requirements

Control Systems Engineer III Qualifications

Education

• Bachelor's Degree (B.S. or B.A.) in Electrical Engineering, Computer Engineering, Systems Engineering, Industrial Engineering, Computer Science, or a closely related field is required.
  

Experience

• 3+ years of experience performing as a Level II or a total of five (5) years progressively responsible experience with Control System Engineering is required.
  Candidates with a Master’s Degree may reduce total experience by 2 years.
  

Licenses and Certifications

• Valid Driver's License at hire and WA Driver's License within 30 days of hire is required.

Background Screening

• Pursuant to the North American Electric Reliability Corporation (NERC) standards, Chelan County Public Utility District #1 conducts background screening of personnel who have access to the District's critical physical and/or cyber assets.

Control Systems Engineer IV Qualifications

Education

• Bachelor's Degree (B.S. or B.A.) in Electrical Engineering, Computer Engineering, Systems Engineering, Industrial Engineering, Computer Science, or a closely related field is required.
• Master’s Degree is preferred.

Experience

• 3+ years of experience performing as a Level III or a total of eight (8) years progressively responsible experience with Control System Engineering is required. Candidates with a Master’s Degree may reduce total experience by 2 years.

Licenses and Certifications

• Valid Driver's License at hire and WA Driver's License within 30 days of hire is required.

Background Screening

• Pursuant to the North American Electric Reliability Corporation (NERC) standards, Chelan County Public Utility District #1 conducts background screening of personnel who have access to the District's critical physical and/or cyber assets.