GitHub Enterprise Cloud with data residency: How we built the next evolution of GitHub Enterprise using GitHub
[ad_1]
Today, We announced that GitHub Enterprise Cloud will offer data residencystarting with the European Union (EU) on October 29, 2024, to address an important customer request and provide our customers with an optimal, unified experience on GitHub.
Data residency and what it means for developers
We’ve been hearing from companies for years that it’s critical for them to be able to control where their data is stored. With data residency, organizations can now store their GitHub code and repository data in their preferred geographic region. With this need met, even more developers around the world will be able to build on the global AI-powered developer platform.
Enterprise cloud with data residency provides enhanced user control and unique namespaces on ghe.com, isolated from the open source cloud on github.com. It is based on the security, business continuity and disaster recovery capabilities of Microsoft Azure.
This is a huge milestone for our customers and for GitHub – a multi-year effort that has required significant time, effort and commitment across the company. We’re excited to go behind the scenes and show how we used GitHub to build the next evolution of the enterprise cloud.
Designing the architecture for the next evolution of GitHub Enterprise
This effort began in summer 2022 with a proof of concept (PoC) and participating teams on GitHub. We carefully considered what architecture would help us succeed. After trying different approaches, we decided to develop the new offering as a feature set that extends the enterprise cloud. This approach would allow us to be constantly in sync with the features on github.com and provide the performance, reliability and security our customers expect. When it comes to hosting, we leveraged Microsoft Azure’s scale, security and regional presence to create a reliable and secure product with built-in data residency without having to build new data centers ourselves.
Since we are the home of all developers, developer experience is crucial to us. We recognized early on that consistency was important and sought to minimize the differences when developing for Enterprise Cloud and Enterprise Cloud with data residency. To this end, the architecture of both is very similar, reducing complexity, risk and development costs. The deployment model is known to our developers: it is based on GitHub Actions. Additionally, changes to github.com and Enterprise Cloud with data residency are deployed every minute as part of a unified pipeline.
To achieve this, we had to organize the work, change our build and deployment systems, and validate the quality of the platform. We were able to do all three of these using GitHub.
Organize issues and projects with GitHub
We used GitHub to organize the project problems And Projectsexploit multiple views to effectively drive work across multiple projects, 100+ teams, and 2,000+ issues. Different stakeholders and teams could use these views to focus on the information most relevant to them. Our talented technical project management team helped coordinate updates and leverage the filtering and cut Ability of projects to present continuously updated information for each milestone in an easy-to-understand manner.
We also used upcoming features like Problems hierarchy to help us understand connections between problems, and Problem types to help clearly classify issues across all repositories. As we leveraged these features internally, we were able to provide feedback to the teams working on them and refine the final product. Stay tuned for future announcements on issue hierarchy and issue types coming soon!
All of these powerful features helped us keep the initiative on track. We were able to clearly understand potential risk areas and work with multiple teams to remove blockages and complex dependencies, effectively advancing the project over multiple years.
Building an enterprise cloud with data residency using GitHub
GitHub has always been built with GitHub in mind. We wanted to continue this practice to be successful with the new data residency feature. To this end, we continued leveraging GitHub Codespaces for development and GitHub Actions for continuous integration (CI). We’ve also added deployment targets for new regions. This created a development, testing and CI model that required no changes for our developers and a delivery process that was tightly integrated into the existing flow.
We have already discussed ours Deploy and then merge Model where we deploy branches before merging them into the main branch. We expanded this approach to include successful deployments to Enterprise Cloud data residency targets before changes could be merged and considered complete while still leveraging what existed GitHub merge queue. A visualization of our monolithic deployment pipeline is shown in the image below.
We’ll start deploying to environments shared by GitHub employees. This includes the internal environment for Enterprise Cloud with data residency, which is discussed in more detail in the next section. Because we use GitHub to build GitHub every day, this step helps us identify issues as employees use the product before they impact our customers. After automated and manual testing, we proceed with the rollout on Canary. Canary is the name for the phase in which we configure our load balancers to gradually forward an increasing percentage of github.com traffic to the updated version of the code. Additional tests take place between the individual phases. Once we have successfully deployed the updated version of github.com to all users, we deploy and validate the Enterprise Cloud with data residency in the EU before completing the process and merging the pull request.
By ensuring that all deployments are successful before merging them, changes are deployed synchronously across enterprise cloud environments and monitored effectively. Note that in addition to deployments, we also use feature flags to gradually roll out changes to customer groups to reduce risk. If a deployment fails to a target, we completely revert the change. Once we detect the error and are ready to redeploy, the whole process starts over with the merge queue.
Finally, to ensure consistency across teams and services, we built automation to generate deployment pipelines for over 100 services so that each service is automatically deployed in a consistent order in the new environment as new targets are introduced.
We ourselves use the enterprise cloud with data residency
To create the best possible product, we also prioritized the use itself and created an isolated environment for this purpose. With ours GitHub migration toolswe have postponed the daily development for the team we are working on GitHub Enterprise Importer adapted to this environment and invested in updating our build, deployment and development environments to support working in the data-resident environment. Since its inception, we have deployed this environment over 8,000 times. This gave us invaluable feedback on the experience of working with the product with issues, pull requests and actions that we were able to address early in the development process. We were also able to iterate our status page tools and internal Service Level Objective (SLO) process with the new environment in mind. The team still works in this environment today and carries out over 1,000 promotional orders every month. This is a testament to the stability and quality we have been able to deliver and our commitment to this feature.
What’s next?
We’re proud that we’ve been able to evolve the Enterprise Cloud to provide data residency and use GitHub to organize, build, deploy and test. We’re excited to unlock GitHub to even more developers and give you the opportunity to experience what we’ve created, starting October 29, 2024 in the EU, with additional regions on the way.
If you are interested in Enterprise Cloud with data residency, Please join us at GitHub Universe 2024 to learn more and hear from other companies about how they have used this to accelerate software development and innovation.
Written by
[ad_2]
Source link