Star Citizen: Around the Verse – Super Hornet & Multi-Region Servers

Closed Captioning provided by Sandi Gardiner (SG): Hello and welcome to
Around the Verse, our weekly in depth look at the development of Star Citizen, I’m
Sandi Gardiner and joining us back in the studio today is Game Director, Chris Roberts. Did you have a productive time overseas? Chris Roberts (CR): Yes I did, but I’m glad
to be back and the weather is a little nicer here, but it was a great trip. I spent some time with both the U.K. and Frankfurt
teams and pushed ahead on some of the new systems coming online as we continue towards
the release of Alpha 3.0 and Squadron 42. These systems are going to go a long way to
support Star Citizens vast map size and huge array of player options. We’ll be showing off a few highlights when
we check in with the studios during the up and coming weekly ATV updates in the next
few months. SG: Now you did mention the best part from
last week’s newsletter that you got to judge a bakeoff. CR: Yeah [Laughs]. It was actually quite good. I had a big sugar rush, but it was the very
first Foundry 42 cake bakeoff. The entries we got were spectacular, I’m
not sure I got that much work done later in the afternoon, but well done to everyone there. I didn’t know we had that much baking talent
in our U.K. office. SG: There you go. The January monthly report was posted to the
website last Friday and has a ton of information about all the progress that was made on Star
Citizen and Squadron 42 heading into the release of Alpha 2.6.1 patch. CR: Yup and thanks to all the Evocati, subscribers,
and backers who have been testing the new Mega Map and multi-regional servers. The patch is expected to go live in the next
few days so watch out for that. SG: With the new patch out we wanted to give
everyone an opportunity to get some new ships as well so we’re having our first ever Valentine’s
Day Sale right now. SG/CR: Awe. SG: Sweet. CR: So on top of that we’re having a Valentine’s
freefly event through Sunday where you can check out one of our multi-crew ships, the
Constellation which is sort of the grand daddy of the multi-crew ships in the game. So make sure to tell your love ones and visit
the site for more details. SG: Along with 2.6.1 we’re also launching
our new chat and forum service Spectrum to the main site for everyone. CR: Yeah this new service is bringing some
impressive communication features which should make it even easier for backers and devs alike
to share ideas and interact. I’m really excited by Spectrum. SG: Now it’s time for the weekly studio
update so let’s check in with the U.K. office to find out the latest developments from the
Foundry 42 team. [2:29] Studio Update Phil Meller (PM): Hello and welcome back to
Foundry 42 in the UK for our studio update. I am lead designer Phil Meller… Mici Oliver (MO): and I’m QA tester Mici
Oliver. PM: Our team here in Wilmslow continues to
grow and we now have over 200 staff here and while the team comprises nearly every aspect
that goes into making Star Citizen and Squadron 42, here are some of the highlights of what
the team has been working on lately. MO: Our props team have a few components and
ship parts to show us including a shield generator, a storage console, and a mining module. [Shows off a Yorm medium-sized shield generator,
looking finished or close to it; a storage console in whitebox; and a mining module rather
fall along in the process.] PM: So, our VFX team have been having fun
blowing up ships again. Some people get all the fun. [Shows off Dragonfly damage states.] PM: And another example of the amount of detail
that goes into our level environments is this in-fiction magazine cover from our UI team. [Shows off Hitbox magazine cover featuring
Star Marine.] MO: As the game expands, and becomes more
complicated, our design team uses some awesome internal tools such as this map of the Stanton
system. We thought you might like it, so let’s take
a look. [Shows off Stanton system focusing in initially
on Microtech before zooming out to see the entire system.] PM: The artists are always adding detail and
character to all areas such as asteroids, nebulas, modular satellites, and stations. [Shows off intricately created asteroid, nebula,
a variety of large satellites, and an artist creating a modular space station.] MO: So there’s a look at some of the things
that we’ve been working on. PM: And now, back over to your hosts. [5:46] Back to Studio CR: Thanks for the update Phil and Mici! Fun to see that map sneak peek in there and
some of the progress happening as we expand out the Stanton system for 3.0. There’s going to be so many things, and
so many little areas, and so much content. It’s going to be pretty cool. SG: And that magazine cover was a great example
of the lengths our artists and narrative team go to, to expand out the immersiveness of
the game. CR: Yeah, especially with the new camera modes
allowing players to zoom in on the environment and take in all of the little details. I’m always amazed by all of the great shots
that I see online that people post of their play sessions. SG: Coming up now, we have another edition
of our ongoing series of Ship Shape. CR: Yes, this week we’re going to take a
comprehensive look at one of the ships that started it all – Anvil Aerospace’s Super
Hornet. And, for 2.6.1, we have a new version of it
which partly we’ll show in this little piece. [6:39] Ship Shape: Hornet to Super Hornet Ben Lesnick (BL): If there was a single signature
Star Citizen ship it would have to be the Anvil Hornet. The Hornets name sake was the F-36 Hornet
from the original Wing Commander. CR: For me when I started to do Star Citizen,
obviously all my past games loom large and one of the inspirations for Star Citizen and
Squadron 42 was my Wing Commander series as well as Privateer and Freelancer and so I
felt like, “Well let’s start with an iconic fighter”, and one of the most iconic fighters
in Wing Commander which was the light fighter that you started in. The naming of the Hornet and quite a few other
Wing Commander ships were always based on historical precedent because I took a lot
from World War 2 or present day aerospace and military facts and so the Hornet was a
light fighter in Wing Commander and so when I started on the Star Citizen I was like,
“okay we need a ship, I need a ship to build, but I can fly around and I can sort of test
out all the different things in terms of the flight model and how the look of it it’s
going to be and how tactile, how visceral it’s going to feel.” So I thought, “Well let’s come up with a
fighter craft and call it the Hornet” as a nod to what I did in Wing Commander and
a nod to historical ships whether it’s the F-18 Hornet today or back in World War 2 Days. So that was essentially the inspiration. Originally I had given a brief to Rob McKinnon
who is a Concept Artist that designed the Hornet and has worked on a whole bunch of
stuff in Star Citizen. You see a lot of his designs ultimately in
the game and I told them, “Well it should be a light fighter” and he would come back
with these really cool designs, but he sort of gunned it up. He put this big ball turret on the top, and
guns on the front with a cannon turret and it was like, super heavily armed, it really
wasn’t a light fighter, but I liked the design so much that I was like, “Okay you
know what, it doesn’t need to be a light fighter, this could be a medium or heavy fighter
and we’ll call it the Hornet and it’ll be the base standard fighter for the UEE navy”
which was the fiction that me and Dave Haddock who’d come up for the universe who you would
fight on the side of in Squadron 42. Dave Haddock (DH): The idea was that it was
originally released in 2806. It was basically a replacement frontline kind
of carrier dogfighter. The idea was the Avenger used to be their
premiere dogfighter and then the Hornet came out and blew them out of the water and became
the ubiquitous of the UEE navy. CR: Once I had the design, I handed over to
design with that, Ryan Churches design of the Bengal Carrier and also of the Vanduul
Scythe which was done by Jim Martin. They were all handed CGbot when I was doing
my early tech demo and prototype of what would eventually be carrying what I showed and the
little movie we put together that hannes put this cinematic, five minute cinematic together
and we headed to CGbot because I didn’t really have a team, it was a sort of loose
affiliation of people helping me put this demo together and Sergio Rossi who used to
work for me at Digital Anvil as an Art Director and Origins Art Director had a art company
called CGbot which is mostly based in Monterrey, Mexico and had some people in Austin, Texas
and he’s like. “Oh I’ve got some excess capacity and
I’d love to help you out” and so we negotiated a very nice rate that he gave me because obviously
this was very speculative and I said, “Well okay here’s the things i Need to build”
and I gave him the Vanduul Scythe, I gave him the Hornet, I gave him the Bengal Carrier
and they also did characters so they did the pilot character that you saw in the opening
cinematic that you would have also seen in the cockpit of the Hornet and they also modelled
an early version of the Vanduul. They built those and then once I got those
assets I, myself put them into the engine and was extending the engine to handle assets
of that detail and fidelity with the articulation and animation that I wanted to see in the
game because for me this was about proof of concept and visualising the feel, the tactical
sense that I wanted the world of Star Citizen and the game itself to have and I think it’s
one of the strengths we have is that our ships feel very real, they feel like you walk around
them and you feel like you get into the seat and all the switches and all the bits work
and they feel proper. It’s not just a cockpit that’s put in
front of you, it’s not just an exterior view of your ship that the insides aren’t
really figured out and so the Hornet was really the first one that did that and sort of showed
what that could feel like and I think that was one of the big reasons why people responded
to it. Chris Smith (CS): The time first when I got
the Hornet in my lap was when we decided to do like the first visual upgrade pass on it. It was already about two years at that time
and then so the decision was made, we had to upgrade it a little bit. What I mainly did was take the landing gear
components and stuff like that and uprezzed those. I didn’t do much on the mainship at that
point, like the main hull, the wings, and all those things. The main features weren’t upgraded at that
point, maybe the materials and the attachments and stuff like that. Mark Abent (MA): Back then we didn’t have
a lot of the technology that we have now. We just had some asset that an artist made
and we just somehow had to make it work. So we had Chris Smith’s beautiful Hornet
and then we had our hacked up versions over here. So we take his stuff, put it in, get it working
how it want it to because we were still prototyping stuff so any one of us would be in that CGA
mashing stuff together and to get what we needed it to be so we could go, “Alright
Chris Smith, this is how we want it, take the beautiful Hornet and rearrange the hierarchy
to how this should be”. So one day we come in and we have guns attached
and then the next day we change the whole hierarchy and it would stop working. Then the following day we’ve got the parts
breaking off and then we would have to redo parts of the item port system so we could
get the hierarchy chain just there. So then maybe next week we get the damage
working and the hierarchy working so you could blow parts off, but now you can’t shoot. So it was kind of like that iterative process
and then once… it took about a month to get everything there, it was a little bit
buggy, but it was like, “Hey this is working, this is pretty cool”. CR: Obviously after we had shown it at the
2012 sort demo, prototype, the Hangar Module was the first time that people actually were
up close, get to see it. Now we worked pretty hard to get all the base
ships like the Aurora, 300i and obviously the Hornet into the Hangar Module which we
debuted at Gamescom back at 2013 which was a pretty cool moment and then not long after
that we debuted a commercial for the Hornet which Hannes had put together which was still
one our cooler commercials and it’s… A Hornet pilot being chased by various ships,
an Aurora, an 300i, and then he comes back home and he revealed that he was bringing
ice cream back for his Significant other. It was really fun and then we sort of had
this idea and I think this was one of the very early cases of coming up with idea of
variants where we would have specialty versions of the ships. So there was a Tracker, or a Ghost and the
Ghost was a stealth version and the Tracker was the AWACS version. Then the Super Hornet, that was the version
of the F-18 Super Hornet that you have today where it’s a two seater, you’ve got a
rico in the back. Maybe that person controls the gimballed turret
or could control missile targeting and that’s certainly is going to become a much cooler
role as Item system 2.0 comes out on the newer stuff, but that was the next update of the
Hornet and Chris Smith sort of had taken the base Hornet and was working in these new variants
that we had done designs and sketches on. CGbot helped out a little bit to add parts
of it like the dome on the Tracker and stuff like that. So we updated it for 2013 and the Hanger Module
and subsequent to that we started to roll out PBR: Physically Based Rendering which
is a whole different, which is what the engine uses now and has been for a couple years,
but it’s a way that the materials look much better, much more realistic, reaction to light. So metal really reacts like metal, Wood reacts
like wood, concrete reacts like concrete. So Chris had then took the Hornet and did
a pass to bring up the materials to the PBR level and it stayed at that point until recently
when he took a pass to take all our Hornets starting with the F7-C and then the Tracker
and the Ghost and now with 2.6.1 it’ll be the Super Hornet to our new modeling with
custom normals and poly mapping and UV2s and our special damage shader that we have. CS: After that first upgrade pass it should
pretty much remain unchanged aside from the PBR upgrade when we got that online and then
it was decided that we do like another version of the ship, like the new 2018 BMW, it was
supposed to be the next generation ship which was the F7-A and lucky for me I was tasked
with doing that and that was great, it was fun since I had been involved with the Hornet
for so long it made sense. I knew a lot about the ship already, I was
very familiar with it. So yeah I took on the F7-A and I sort of designed
it up from the ground up and I kind of decided all of the things that bothered me on the
original Hornet I sort of wanted to fix on this new one so that’s what I did. So after the F7-A, the Mark Two was complete
and I moved right onto upgrading the Mark One yet again, but this time it was a full
upgrade. I wasn’t just upgrading pieces here and
there, it was sort of rebuilding the whole ship with smart normals and the proper material
PBR values and everything and so it could hold up with all the other ships that we’ve
been coming out with for the last year and a half. Since I had done the upgrade to the base Mark
One Hornet, they all share parts so the change actually affects all the variants automatically
which is great, it saves us time because it reuses the wing and the cockpit and everything
like that. All that was needed was some material tweaks
and colours and stuff like that. After that, it was the Super Hornet and the
Super Hornet was a little bit of a special case because A, it’s a two seater and B
al ot of the parts aren’t directly shared with the original Hornet. There’s some tweaks and additions to the
Super Hornet that the original base Hornet doesn’t have. Kirk Tome (KT): We felt the Hornet line was
due for an update and so we wanted to bring the Hornets up to the current standard of
our latest batch of ships. We wanted to improve them both visually and
technically. That did involve things like streaming the
design shape. Taking a look at rebalancing the ships and
changing their weapon loadout for example, their missile payload and also figuring out
whether the current setup for the interior of the cockpit was going to be useful for
our future multi-crew and item manipulation system that we’re applying in the near future. We’re updating the damage tech on the ships
so that they use our current UV2 damage so that they’ll blow up really nicely. Patrick Salerno (PS): The old damage system
used to have 25, 75, 50, 100 states. You would shoot the ship, after a certain
amount of health, does a model swap, 100%, blows off. It was very cool damage, but it was very labour
intense. We’d actually have to do a lot of hand modelling,
each piece would end up looking very similar after awhile. MA: If you picture my hand as the Hornet,
a horribly drawn Hornet. Like each of these fingers would be parts
and we should be able to shoot them and blow them off. In theory it sounds like an easy peasy thing,
but we had it much more complicated than that. You could blow up this part of the thumb,
followed by this part. So I could shoot here and it would take the
whole thing away or shoot here and take the whole thing away. PS: When the new shader came along, basically
now we just blow off huge chunks. So wing, wing debris and when I’m taking
a ship and I’m blowing it apart, I’m saying to myself, “which pieces detach from where
and I’m setting up the proxies, I’m setting up the shields, I’m making sure things still
animate and detach”. If there’s a turret or gun under the wing
I have to make sure the helpers are attached to that piece of geo so it goes flying off
with the debris and the destruction. MA: It’s basically applying this streamlined
texture around the whole ship and when you shoot it, you get these nice glow effects
and decals that happen anywhere on the ship. With that we have this like underneath skelton
carriage where if you shoot it so much, you’ll start seeing the bare bones skeleton of the
ship. PS: So now that I have pieces detaching, I
have basic shader damage, I also need to go in and make sure that there are effects setup. Those basically represent a bunch of little
cubes in that and they’re floating around and it kind of seems like C4 charges right
, they explode and then you get the UV2 damage. So there’s burns and there’s wear on all
the pieces that are exploded and melting and once those detach, it creates more randomized
damage. So I can hand place squibs so even though
say two wings detach on the same spot, there’s different charges in different spots and it
looks more varied. KT: The main impact is going to be visual. You will see that the lines are a lot sexier. You’ll see that when you die, explosions
are more satisfying and it doesn’t feel as outdated as it used to. CR: It’s really amazing to see all these
back to back and there’s quite a difference in terms of detail and quality and the amazing
thing was back in 2012 I thought, “Oh my god, this is amazing, I don’t think you
could get something more detailed or cooler in a game back then and I sort of look at
how we move along every year and how better we get and I’m always in awe at the ability
and talent of our artists. For us to build this world it feels that tactile
with these ships that feel real, they don’t really feel like a fake or a digital ship,
they feel like a real ship and that’s our goal is to lose yourself in the cockpit of
one of these ships and you know, fly in space and live that fantasy. [22:32] Back to Studio SG: That really takes me back to when we were
putting together the original demo for the Kickstarter. Do you remember that? CR: I do. Well, I don’t remember much sleep from it. Anyway, so much of Star Citizen’s development
history is represented in the tech and artistry that can be found in the Super Hornet and
the other Hornets. So, to see where we started and where we are
now makes me really proud and even more excited where the project is going to go next. SG: Speaking of what’s next, we’ve got
a really interesting piece for this week’s deep dive featurette exploring some of the
new server improvements that our engineers integrated into our network code for alpha
2.6.1. CR: Yes, so have a look at how our new multi-regional
server support and enhanced cloud architecture is allowing players to connect faster and
easier no matter where they are in the world. This will be something we will be continuing
to work on and roll out to other regions as it beds in and works well. [23:25] Behind the Scenes: Multi-Region Servers John Erskine [JE]: Hey guys! Ahmed Shaker [AS]: Hi John. Mike Jones [MJ]: John. JE: I’m here with Mike Jones and Ahmed. Today we’re going to talk about some of
our server architecture and the cloud and some things that we’ve been working on from
the operations and the Dev Ops side lately. Let’s start off first by talking about what
we have today. Most people don’t know very much about what
it takes to publish a game like Star Citizen. We’re in an alpha phase now and the game
is available to anybody who is a backer. We have regular free fly events where even
people who aren’t backers can check out the game. So, tell us a little bit about what is involved
in actually running a game like Star Citizen today. AS: Almost everything we use to serve the
game or the platform to the players is run on public clouds and that has been almost
always the case, as far as I know, and we have been trying different public clouds to
see the perfect offering for what we need for the long term and also the short term. However, I believe IT under Mike Jones had
been running our own internal clouds here as well. So, we still utilize the cloud technology
whether it’s public or private on hardware owned by us. MJ: The main thing Ahmed’s talking about
is where do we put the servers and the server – or services – that we connect to when we
play Star Citizen could be anywhere. It’s really just a computer on the internet. So, we could host that here in our own servers
– this is the way we used to do it in the old days. The problem with that model is that first
you have to acquire a lot of servers and that’s a pretty big expense whereas with the cloud
or hosted rental market we can only rent or buy what we need and then we can turn it off
and stop paying for it if we’re not using it. With dedicated hardware we have a massive
overhead in maintaining that equipment whereas if we’re using someone else’s, we don’t
have to worry so much about that. If a fan goes out or a CPU or a power supply
goes out in a virtual environment, these systems are designed to automatically roll over to
the next available machine. So, this gives us a tremendous amount of flexibility
at a much lower price point than if we had to acquire everything ourselves. One of the things that I think is really important
to point out, and I hope that Ahmed can explain to you, is the additional flexibility we get
in the network. If we just hosted here, let’s say in Austin
or at any of our studios, the players would be connecting to that one place. What we can do now by utilizing servers located
throughout the world is that we could put servers anywhere we need them, wherever the
players are, if that makes sense to do. JE: Ahmed, do you want to tell us a little
bit more about the services that actually comprise Star Citizen today. AS: Absolutely. So, the game itself needs – any client that
any player has needs another component. The DGS, the dedicated game server, we always
call it DGS. However, you cannot have this connection between
the player and the dedicated game server he wants to play with unless you have a lot of
backend services taking care of a lot of business logic that the player needs. For example, the player needs to be able to
declare his request for the game he wants to play. He wants to say that he wants to play with
his friends, communicate with them, chat with them and these are the services that are being
developed here in Austin by our backend services team – Jason Ely and Tom Sawyer. Currently what we have, we have our layout
– our VM mainly full in three main types. We have a hub server or a hub VM and a core
VM and a game VM. The game VM is just a virtual machine that
we acquire from our cloud provider, in our current case it’s EC2, and you would load
it with DGS instances according to the amount of load that we decided on – how many instances
would run on how many CPUs, how much memory, how much network, and they all get connected
to the core services. The core box would have several services,
most importantly we have GIM – the Generic Instance Manager – a lot of backers already
are familiar with the name and know what it does, and we have the PCache, Persistence
cache. Persistence cache sits before the Persistence
database service that writes direct to the database, taking all of the traffic from all
of the dedicated game servers and pushing it down to the database in more of a cubed
organized manner rather than having all of the game servers hammering one single service. The current model that we have, we can have
as many hubs as we need, we can have as many game servers as we need, but we have only
one singleton of every one of these services and that’s the current model. Because usually, due to my experience in startups
you go out with proofing your concept, then having your MVP: Minimal Viable Product, make
sure that’s what you need, and then you start moving from there. So currently, the backend service team are
working on revamping and redesigning all of these singletons into so many stateless, easy-to-scale,
services that we can have whatever we want and cope with whatever load we get and that’s
the main point that we are in the cloud. One of the main great things about being in
the cloud is being able to scale whenever we want. Back in the days when you were trying to market
the cloud, you always used to bring up examples from Black Friday. You have a shop that sells flour or sells
toys, year-round you can be fine running on a single VM – on Black Friday you need 200. That’s when you go to the cloud, and we
have the same concept because we need to be always able to provide resources and game
servers to the players who play – no matter how many of them decided to play, no matter
where they are. JE: Do you want to tell us a little bit about
how we’ve added functionality or added features over time and where that gets us today and
I am particularly interested in understanding from your experience the level of complexity
we have in Star Citizen today, already, compared to a lot of other games. MJ: One way I like to describe that is a physical
way that you can visualize. When we were developing the hangar module,
of course we had fewer people, and everything we did as a company came together in one computer
– a tall one but not a very big computer – and it sat under one guy’s desk and we got one
build a day. The needs of the publishing increased so rapidly
that this one computer wasn’t going to be enough and when we took a look at this we
realized, we would need a room full of computers to do this. So, this is what Ahmed talks about when we
developed our own internal cloud. So, we’ve built a virtualization infrastructure
where we can actually expand, much in the way the game server is run, we expand an internal
build system that increases dynamically based on the needs. JE: Tell me about what you’ve been working
on lately. This is a major new feature that all of this
infrastructure and architecture that we’ve talked about has gone through multiple generations
and iterations and really set us up for something that you’ve been working on now with other
engineers. Tell us about it. AS: So, since December we started looking
at – we released Star Marine and now we have a lot of players who want to play FPS and
a lot of players have been asking us – where is the multiregion work? When we can we be able to play on servers
closer to us? So, we all agreed this is the right moment. We have an FPS right now, so let’s have
multi-regions. So, since December and including January and
a little bit of February we have been working on the multi-regions servers to be able to
have servers in as many regions as we want, without the player having to disconnect but
he has to move from one region to another, because they all share a single database. So, the same player can play in the EU and
then a few minutes later can switch and go back and play in the US and he’ll find the
same information that he needs. As any new feature that we come out with,
as I was just explaining, we try to come out with a certain delivery between the PoC, a
proof-of-concept, and an MVP, a minimum-viable product, and that’s what we have right now
and I believe we have selection for the regions that we will start with and we have building
or editing our automation tools, SaltStack, to be able to code with different regions,
different naming, different availability zones which is a concept that happened inside AWS
and how we can add resilience while we are adding availability at the same time so we
always know if something went down over here or have a placement for it over there but
also we are trying to get as close as we can to our player base. We have our current region in Virginia, North
Virginia, US-East one and we will be adding Frankfurt and we will be adding Sydney. It’s been a lot of fun, every single minute
that we spent on this project connecting these regions together, making sure that everything
worked correctly. It was kind of different from any other changes
that we did because we know that this will really enhance the way it’s experienced. We know that the players will feel that. There’s a lot of work that we get to do
but we’re like, ‘We just did something really cool but the players won’t really
get to notice it,’ but this one we really hope that you notice it, you like it and we
can embrace it and expand it on it too. JE: One thing I know about the multi-region
work is that there’s really two levels to this experience from the players perspective
of playing in the different region or being able to choose a region. One piece of it is where the game server is
located. This is the DGS that you talked about, and
the other piece which is invisible to the player is where the HUB server is located. AS: Yes JE: So from my understanding is that as we
roll out to these different regions that players will always connect to the HUB server that
closest to them. AS: Absolutely. JE: Which gets them into our network at that
point instead of going over the open internet and then from there they can play on a dedicated
game server that’s either located in that same region or that DGS may be located in
another region. AS: So the way this feature is going to overlap
utilizing the technology of using a new extension of DNS that would allow the DNS server to
know a little bit more about the location of the requester, we would be able to use
a service like Cal53 to send players to the closest load balancer to them and the load
balancer it would take them to the most suitable HUB server that you have. The moment you get in this HUB, you will get
marked with a region ID, that’s the NID you walked in with and the matchmaker will
respect that choice. You came from region ID that means Europe,
then you’re going to get sent to a game server in Europe. You want to go play with your friends, make
your selection then go play in the US. So Hopefully this will be rolled out really
soon and players will enjoy it and we’ll expand on it and listen to all the feedback
and try to make it much better. MJ: Ahmed makes it sound so simple. JE: I know right? Makes perfect sense. AS: [Laughs] MJ: We’ve been working on this for some
time and it has been a fun project, but there is a tremendous amount of network engineering
and network security that goes into this and also performance tuning that has to be considered
for this to work. JE: When you said a minute ago, “Well we
worked on this in December and January”, but it’s also the case that if it wasn’t
for the 18 months that came before it that you couldn’t do this in two months. AS: Absolutely, absolutely. That’s where experience kicks in and that’s
how you learn the hard lessons from before. You have to play everything ahead, you have
to make that you have sure areas where you can expand and then get surprised that you’re
going to leave that current region you work in and if you want to talk a little bit about
technical, AWS the largest organisational body we have is called VPC: Virtual Product
Cloud. The Virtual Product Cloud cannot go across
regions so you cannot have one single VPC which is a cloud and by the cloud meaning
it follows one single network CIDR like, it’s a single internal network that you can divide
in between, but you cannot only make a VPC have the same IP address subnet or the same
networking between two different regions. So you have to make sure that you make your
own tunnels and you have to make sure that your VPCs were aligned correctly so that they’re
not going to collide, you will get to know whose coming from what, who needs to what
and no other traffic will not get in the way and also be able to recover because definitely
while you being in the cloud is such a great thing, it’s an amazing thing, but always
design for failure, everything will fail. You have neighbours, you have a lot of very
complicated stack beneath you to know what’s going on. As Mike was explaining, the disks might go
bad or anything could happen so we have been working on that so that even if we lost our
tunnels we can recover in a few minutes from a lurch on it and make sure that we are delivering
that traffic needed from the distant game servers to the core service that runs the
matchmaking or database and all of that with the best achieved latency and the most attainable
bandwidth that we can attain between these different regions. I think we start to kind of like, was a big
stretch, we’re going to Sydney, we’re going from Virginia to Sydney and we’re
going to Frankfurt. Frankfurt is actually central in Europe when
you look at it. We could have one in Ireland which makes it
a little bit easier for us, but we said, “Okay, let’s do it’. So we’re going Sydney, Frankfurt, then Virginia
for the first rollout and all the networks are laid out and we are ready to expand, I
believe there are 14 to 15, we are ready to expand in every single region of AWS when
we feel we need to. JE: Well that’s perfect for my last question
which is to say, where are you going from here? What do you see on the roadmap after this? What does this set you up for and what do
we expect to see in the future. AS: Yeah I mean when it comes to all the regions
I’m not sure If can state all of them? I think I can. They have Ohio, Oregon, California, Ireland,
London, Sydney, Mumbai, Sao Paulo Brazil, I believe Tokyo Japan, Seoul Korea. So yeah, skies the limit and they keep adding
more regions, I believe they’re going to have a new region in France and one more in
China soon. So as you mention, the way it works because
we are here to build this project, we always try to lay our foundation and the support
that we get from you and Mike that we can take our time to build our own foundations
and techs so we can capitalise on it in the future and be able roll quick features, that
what we have been doing. Right now going to any region is not going
to be of any limit to us, and once we’ve rolled out more regions, this year is going
to be a little busy for us because we have a lot of… As much as we were talking about how the backend
services started with having a proof concept and moved then to MVP and these are going
to get revamped, we have a lot of infrastructure elements that we need to look at. We have a lot of areas that we monitor certain
aspects, we need to group some of them, we need to diverse some of them. We need to look more longevity solutions,
we can scale it out because the way you look at it, we need to be ready to have 10,000
VM’s reporting at the same time without crippling us and that’s the reward that
we get from having the DevOps department early on. I hope this answers the question. MJ: Every year we say the same thing, but
this one really feels we’re making some big headway. The automation systems are done now, we feel
like we can scale with a very high reliability. We have far fewer panic nights where we’re
staying up all night trying to solve problems. So definitely the work we’ve put in has
paid great dividends and so now with expansion into multi regions, it just gives us more
that we can do. JE: Cool we’ll I’m really proud of the
work that you and your teams have accomplished and I’m really happy that we have a chance
to share this in more detail with our fans and I look forward to all the great things
that are yet to come. MJ: Thank you John. AS: Good times. JE: Thanks guys. [40:20] Back to Studio CR: Really happy we got take a deeper look
at the team. They are often the unsung heroes of the project
because what they do is critical to making Star Citizen work, but it’s not sexy as
building a spaceship or making environments or a character and there’s been a ton of
effort put into implementing multi-regional servers and some of that backend matchmaking
so it’s great we got to highlight some of that work. Outro SG: It is, well that brings us to the end
of this episode of ATV. As always we’d like to thank our subscribers
for contributing to all of our behind the scenes content. CR: Yup, thank you guys, and I’d also like
to express our continued thanks to all of our backers out there. You do so much to support Star Citizen’s
development. We’re trying to do something unique with
this project and all of you are a large part of that. SG: Yes you are. We’d also like to invite you all to join
us tomorrow at 12 Pacific for the latest Star Citizen Happy Hour stream to watch some live
game and discussion. CR: Yes, so this week, Senior Writer Wiesbaum
will be stopping by to answer some questions. It should be a great way to kick off your
weekend. SG: And do you know what else might make their
weekend even better? CR: Let me guess, a brand new look at what
our artists have been doing with the MISC Prospector? SG: [Cheers] Yes, exactly what I was thinking,
enjoy! We will see you.. SG/CR: Around the Verse!

Leave a Reply

Your email address will not be published. Required fields are marked *