- Performance and Information Technology
- Technology Plan
- Infrastructure Ops Department
Technology Plan
Page Navigation
Infrastructure Ops Department
-
Infrastructure Operations provides a variety of services to support individual schools and the district as a whole, including the overarching IT architecture plan and deployment, implementation and maintenance of email and collaboration tools, network administration, telecom, and wireless integration, management of the data center, MDF/IDF support, oversight of UPS and Citrix/VDI, deployment of applications, desktop and imaging services, and Mac and tablet support.
Virtual Desktop
-
Current Reality
- APS implemented a virtual student desktop environment on a Citrix platform. The district is part of Citrix’s partner for life program, and we have also partnered with a preferred Citrix partner for assistance with Citrix related issues. The student desktop environment supports approximately 12,000 Wyse zero client endpoints to accommodate 50,000 students.
- APS has regular meetings with Citrix to ensure APS is following best practices.
- APS has migrated the control plane (management servers) to the Citrix Cloud to reduce the load on internal resources, both for compute and human support resources.
Gap Analysis
The current infrastructure has the following limitations:
- Software requests typically do not follow any type of standard/process.
- Staff resources: Presently, there is one primary resource that supports the environment for the district. • The initial project was designed solely for students; staff/employees could also benefit from utilizing this solution.
- The outdated Cisco UCS server hardware needs to be replaced.
- Update the security to require multifactor authentication (MFA) for staff to access the environment and not students.
Future Vision
To achieve the future vision of continuing to provide a reliable and safe virtual computing environment, APS will take the following steps:
- Secure enough human capital to support the Citrix environment fulltime.
- Market and communicate that the Citrix environment is a solution for all staff and students that the solution has the capability to run Windows applications and Windows full desktops on Chromebooks and the ability to access internal resources from outside the APS network.
- Remove the Wyse Zero Clients from the classrooms and keep the ones that are in labs and better utilize our investment in Chromebooks by making the Citrix environment accessible from those devices.
- Implement new HPE Synergy servers as the host to replace the outdated Cisco UCS servers that are currently utilized.
- Update the security to require multifactor authentication (MFA) for staff to access the environment and not students.
Data Center
-
Current Reality
APS currently has two Physical Data Centers & One Cloud base Data Center
- Primary Data Center is located at the Center for Learning and Leadership (CLL)
- Disaster Recovery Data Center is located at 56 Marietta Street (Digital Realty)
- Microsoft Azure (Cloud Resource based Data Center)
Servers
(350) + physical servers including (650) + virtual servers
- HPE rack–mounted servers & HPE Synergy Systems (CLL & DR sites)
- IBM PureFlex chassis and blade servers & IBM Thinkservers (BigFix Relay servers 1 per school)
- Cisco UCS chassis and blade servers
- Patching for Physical Security Recording Servers in remote schools and administrative offices
Storage
1+ petabytes of local SAN storage & Azure cloud storage
- Microsoft Azure Cloud-based storage
- Two Brocade IBM Branded SAN Fabric switches, IBM Storwize V7000s & IBM DS3500
- Nimble Storage
Data Center Networking
- HP core and distribution switches, Load Balancers, Citrix NetScaler’s (4) SDX (2 at CLL & 2 at DR) with (4) VPX Virtual Load Balancers for specific areas, (Internet Load Balancing, Identity Management, DNS, VDI, etc.)
- Kemp Virtual Load Master 10GB (2)
- DNS/DHCP
- Microsoft AD-integrated DNS
- Microsoft Dynamic Host Control Protocol (DHCP)
- Content Filtering and Mobile Device Management
- Palo Alto Next Generation Firewall (2)
- Securely Hosted Environment
- Next Generation Firewall
- Palo Alto (2)
Operating Systems
- APS is mostly a Microsoft Windows Server environment, running on a VMware virtual infrastructure for our production business server environment, Disaster Recovery environment and running a Citrix environment on top of Microsoft Windows Hyper–V for our virtual desktop infrastructure (VDI).
- VMware vSphere 6.7U3 with vRealize Operations Monitoring
- Microsoft Windows Server 2012 through 2019
- CentOS 5.x
- Citrix XenApp and XenDesktop 7.x
Servers
- APS is a Microsoft SQL Server database environment
- Microsoft SQL Server 2012 through 2019
Backups
- APS utilizes VEEAM Backup & Replication software to backup various systems in the district.
- TSM for Mail to backup Exchange with weekly full and daily incremental backups
- Currently at 1500-day retention & over 567TB in each storage pool (primary and copy)
- VEEAM Backup & Replication to backup data from selected servers
- Incremental forever type daily backups (30-day retention)
- Currently over 40TB of compressed data is backed up
- VEEAM Backup & Replication to back up our VMware infrastructure
- Incremental forever type backups performed daily. 14 days onsite data & 30 days of offsite data retained
- Currently, over 40TB’s of data stored in Scale-Out-Backup-Repository
Disaster Recovery (DR)
VEEAM is an all-in-one IT Resilience Platform, converging Disaster Recovery, Backup and Cloud Mobility in one simple scalable platform that provides continuous data protection.
- Protects our virtualized VMware infrastructure against misconfiguration, disasters, and ransomware
- Recovery Point Objectives (RPO) happen within a few minutes vs. hours for traditional backup/recovery solutions
- Critical applications such as Lenel badging, secure file transferring, Move-it, Edulog and Security Camera management systems can all be recovered and run from our TeleX DR location
Gap Analysis
- Backup solution for Microsoft 365 Suite of products, including Exchange Online, SharePoint Online, and OneDrive
- Backup solution of Google Suite Data (Google Classroom & Drive)
Future Vision
Storage
- Expand storage capacity by adding additional SAN storage to accommodate future growth needs
- Add fiber channel Nimble Storage array
- Segment development, test, and proof-of-concept workloads off production SAN environment
Data Center Networking
- Create a new virtual lab environment with enough host server, storage, and network resources to migrate all development, testing, and proof-of-concept workloads out of the production virtual environment.
Servers
- Upgrade VMware environment to VMware vSphere 7.x
- Replace outdated Cisco UCS Hardware for the VDI environment with a new HPE Synergy environment
- Upgrade Microsoft Windows Hyper–V environment in the VDI environment to Microsoft Windows Server 2019 and Microsoft System Center Virtual Machine Manager 2019
- Upgrade guest servers to Microsoft Windows Servers
Backups
- Work with the Policy and Governance Team to come up with appropriate retention policies
- Continuously review what is being backed up with the Application Support Team to ensure that all appropriate data is being backed up
- Schedule regular periodic testing of restores from the backups to ensure our ability to recover in the event of a disaster
- Upgrade to the newer versions of VEEAM Backup and Replication
- Decommission old TSM and Spectrum Protect environments
- Implement VEEAM Backup for Office 365 product to handle the backup of Microsoft 365 data (Exchange Online, SharePoint Online, OneDrive)
- Users transition to OneDrive for Documents for enhanced resilience and availability
- Create a private cloud infrastructure for hosting services that can be offered to charter schools
TELECOMMUNICATIONS
-
Current Reality
The current APS telecommunications infrastructure runs on the Cisco platform. All Services except Advanced Quality Manager (AQM) are high availability (HA) and are deployed at both CLL and Telex (Disaster Recovery) sites. The current deployment:
- Telecom PBX switch is Cisco Unified Communications Manager
- Signaling protocol is Session Initiating Protocols (SIP)
- Inbound / Outbound calling is handled by Cisco Unified Border Element (CUBE)
- Call Centers are handled by Cisco Unified Contact Center Express (UCCX)
- Call Recordings for call centers are handled by Calabrio Advanced Quality Manager (AQM)
- XMedius provides the Enterprise Fax (E-Fax) deployment with faxing from the Xerox copiers
- Call Detail Recording (CDR) records are handled by SPLUNK and Side-View Apps
Gap Analysis
- The current deployment of Cisco 7900 series telephones remaining are out of compliance for remediation of TLS 1.0.
- The current UCCX Contact Center makes it difficult to scale into a new call center quickly as the software is antiquated.
- The current Emergency 911 (E911) deployment does not comply with regulations coming out in the next year or so which will cause legal issues if an upgrade is not performed.
Future Vision
- Complete the upgrade of all remaining 7900 series telephone sets to the latest telephone set which is the 8851 set.
- Move to Cloud Based Contact Center solution, to allow scalable contact centers & ease in establishing new call centers.
- Partner with vendor to provide an all-encompassing E911 solution to handle both current regulations/future updates.
- Update the security to require multifactor authentication (MFA) for staff to access the environment and not students.
COLLABORATION TOOLS
-
Current Reality
EMAIL
A hybrid environment, employing Microsoft Exchange Server 2013 on virtual servers for SMTP relay and Microsoft Office 365.
- We have over 65,000 mailboxes in our current environment, running ~150k messages out and ~525k in, each week.
- The migration of users to Office 365 has increased the mailbox default size from 800 MB to 49 GB.
- Proofpoint protection and Microsoft Exchange Online Protection filter incoming messages; Microsoft Exchange Online Protection (EOP) for outgoing messages. Email is now protected by MFA “Multi-Factor Authentication.”
- The on-premises Exchange 2013 environment is monitored through Microsoft Managed Availability tool.
- The Office 365 environment is monitored through the Office 365 Admin Center.
Gap Analysis
EMAIL
- Microsoft Exchange Version – Microsoft Exchange Version 2013 is now in Extended support until April 2023.
- Monitoring
- On premise 2013 environment is monitored through Microsoft Managed Availability tool.
- Cloud is managed through Office 365 Admin Center.
- Documents and Attachments – Version 2010 had limited capability and size limited to 10MB.
- Office 365 size limit increased to 55 MB and protected by Safe Attachments.
- High Availability/Disaster Recovery – Version 2010 requires manual restart, no self-recovery capabilities.
- Microsoft Version 2013 exchange environment deploys self-recovery capabilities on virtual machines.
- O365 Environment is Cloud based housed on Microsoft’s servers with multi-location redundancy.
- Mailbox Size – Version 2010 default mailbox size was set at 800 MB, O365 default mailbox size is set at 49 GB – 99 GB.
COLLABORATION
APS has multiple collaboration tools. In the Cloud we have Microsoft Teams, OneDrive, Zoom, and SharePoint.
Future Vision
EMAIL
- Update 2013 exchange servers to 2019 or build a 2019 server and switch on-premises services.
- Create a knowledge base to better acquaint users with Outlook 365 capabilities.
- Update the security to require multifactor authentication (MFA) for staff to access the environment and not students.
COLLABORATION
- Continue Microsoft collaboration environment knowledgebase to acquaint users with the capabilities of Teams.
- Provide licenses to enable conference calling Teams for all users.
- Purchase Zoom Webinar licenses that will allow district programs to be able to host large participant webinars.
WIRELESS INFRASTRUCTURE
-
Current Reality
- Infrastructure is Xirrus platform, with ~1500 wireless access points across the District, installed in hallways providing wireless in classrooms. The design provides a signal strength of -65db.
- There are two on-site controller servers to manage and monitor the wireless infrastructure.
Gap Analysis
- Capacity & Coverage – The current deployment model does not provide sufficient coverage or capacity to meet current and future business needs, especially with the ever increasing use of mobile devices for instruction.
- Optimized Performance (802.ac Compliance) – The current wireless infrastructure does not have the capability to support optimized performance, a feature of the latest wireless technologies, such as 802.11ac/ad.
- Ease of Management & Monitoring – As the management servers cannot support redundancy and are only available on–premise/over VPN, the current wireless infrastructure doesn’t lend itself to easy management or monitoring.
Future Vision
Recent contract implements a managed services solution to close existing gaps.
- A new design implemented to provide a wireless infrastructure of a 1:1. This will put 1 AP (access point) per classroom, in hallways, office areas, auditoriums, gyms, conference rooms, and other instructional spaces. This scalable design model will provide adequate coverage and capacity.
- Install Optimized Performance Capable APs – The new upgrade will replace the current Xirrus Arrays with Cisco Meraki APs. The new Meraki APs will be optimized for high performance, and are compliant with 802.ac/ad standards. They also feature backward compatibility with older technologies, such as 802.11a/b/g/n.
- Cloud–Managed Service – The new upgrade will also come with a cloud–managed service that will allow access to manage the infrastructure from anywhere over the internet, 24/7 monitoring, and alerting capabilities.
- Student Wireless SSID – There will be a separate wireless network specifically provisioned for students.
- Bring Your Own Device (BYOD) – The APS board has approved a BYOD policy which will allow APS staff and students to bring in their own devices and connect securely to the APS network.
MDM (MOBILE DEVICE MANAGEMENT)
-
Current Reality
APS has made a substantial investment in the purchase of Chromebooks and iPads, primarily purchased as devices for students to use during state-mandated online testing, online assessments, and virtual instructional needs from home.
Gap Analysis
- The current device management platform does not easily support the management of iOS or Chrome OS devices.
- There is a need to push software across the internet to iOS and Chrome OS devices.
- Staff resources: one primary resource each supports the iOS and Chrome OS environments.
Future Vision
- Implement Google Admin Console to manage Chrome OS devices (Google has a proprietary MDM to manage Chrome OS).
- Phase out Lightspeed and move devices to Securly MDS to manage iOS devices.
- Secure and cross-train enough human capital to support the MDM environments fulltime.
- Institute regular meetings with our partners to ensure APS is following best practice.
APPLICATION SUPPORT
-
Current Reality
A four-person team manages application scripting, imaging, and software deployment. Tools used include LANDesk, Microsoft System Center, AutoIt, Orca, AppDeploy (Kace), and Powershell. Requests generated via work order system Nimbus or projects.
Gap Analysis
- Absence of Application Support Request Process.
- Inability to Accurately Monitor Software Licensing – IT does not have oversight of all District software purchases.
- Standardization of Hardware Platforms – The District currently allows sites to purchase hardware independently.
- Not Fully Utilizing Software Tools.
Future Vision
- Develop and implement software acquisition requests and deployment processes for the District.
- Develop standards for requesting, testing, and deploying software.
- Collaborate with Procurement to develop processes that restrict the purchase of software until vetted by IT.
- Allow IT the ability to manage all software licensing for District, allowing better support from Applications Support Team.
- Develop, implement, and enforce hardware standards to allow for better software and hardware management.
- Pair training with application rollouts at the appropriate time(s) to leverage full capabilities of the tools.