We're not. For a company of our age and scale? To claim competitive reliability to the world's most important data center? Risible. We're making fun of the fact that everybody deploys their apps in a random city in Virginia with like 3 restaurants in it. We're not saying they're dumb to do it, or that Ashburn isn't up to the challenge --- it self-evidently is.
(The comment I'm responding to said something about people in glass houses not throwing stones, but was then edited).
You guys may want to rebrand once you get your issues sorted out. Fly is going to have the same problem as btrfs, where folks will have a hard time deciding if it's ready for production yet.
It's comments like this that moved me to claim, in our infra/ops job post, the benefit "The most lovable user base in the whole technology industry, and a company culture that will keep you in touch with them directly."
Fly.io was the darling of the webhosting world amongst techworkers particularly because of the honest transparent communication and blog posts explicitly admitting fuck ups
Raising a bunch of money just isn't a "good" announcement to most of us. When unpopular things have happened with the company, your communication usually directly acknowledges and focuses on the concerns of the userbase. This serves two purposes:
1. It gives you a chance to show concrete steps to prevent specific concerns we have
2. Perhaps more importantly, it shows we're all on the same page and you're actually in touch with how your userbase thinks
Failing on #1 is something you can fix later, but I've never seen a company rebound from losing the trust gained by #2. It loses confidence that future decisions made can also be equally out of touch.
I've never seen a company rebound from losing that confidence
If we're "honest" and "transparent", it's not a conscious strategy to sign up customers. It's just the easiest way for us to write. Meanwhile: we're a hardware-based global hosting company. We've always been externally funded. That's how you build a global hardware-backed hosting company, because servers and colocation and infra engineers are expensive.
There are things that happen that disappoint customers that we care a lot about. Reliability issues several months ago absolutely fit that bill. So does hiring: we paused hiring for most of our engineering roles because we weren't happy with the experience candidates were getting. I don't know how "honest" and "transparent" we are about those things, but I do feel like when we're asked about them, we give straight answers.
There are other things that happen that disappoint people that we don't care so much about. Being unhappy that we took external funding to build out our fleet and improve reliability would count as one of those things. Again with the straight answers: I wouldn't expect us to spend too much time trying to "rebound" from this announcement.
>Raising a bunch of money just isn't a "good" announcement to most of us
Speak for yourself. Announcements of adequate funding inspires confidence fly will have longevity and resources to scale. Other concerns can be tackled in other blog posts, but nothing about the announcement of funding should inspire less confidence, what sense does that make?
The issue is not raising money, precisely, the issue is raising money by selling large amounts of controlling equity (as is usually the case for VC funding and IPOs, with a few notable exceptions). The purchasers will expect their money’s worth, and they will (eventually) not hesitate to exercise the control it bought them in order to get that.
VCs in particular will expect much more than their money’s worth, or (most of the time) for the company to die trying to earn it, and they won’t hesitate to force the latter in pursuit of the former. This increase in variance is literally their investment model.
So if VCs expect their investments to fail most of the time, and I see a company take VC money, I must have very strong reasons if I’m to disbelieve the VCs and expect the company to succeed—and I’m not following a venture investment strategy when I choose a tech supplier.
IPOs are somewhat milder in their intrinsic character, but they still usually lead to professional boards of directors and eventually professional execs, where “professional” usually means “no subject matter expertise”. And hardly anybody runs a public company on customer satisfaction nowadays, unfortunately.
So when I see controlling equity sold, my outlook changes from “will this go to shit?” to “when this inevitably goes to shit, will it be in the distant enough future that I don’t care or can plan for the loss?” (that being maybe a decade or so for infra, or implausibly long—like the rest of my natural life—for anything involving a community).
As usual, the question remains—why does none of this seem to be priced in? I don’t know.
To be fair (this comment is mostly a joke), Ashburn/Loudoun County is pretty nice for a suburb/strip mall hell. I've had some really good food in that area.
So you fuss at your users who have valid complaints about the availability of your product? It's a valid complaint when the uptime of a service, program, or feature running is critical to your business.
I'm really concerned that Fly's employees like yourself think availability and service uptime isn't important. I think I'll be avoiding Fly on this alone.
This is neither here nor there, but every time I read a comment like this, all I can think about is that scene in The Office where Dwight Schrute is screaming into his cell phone "Are you saying you invented paper?!"
Yeah this is pretty funny - if you speak to PR before posting something, it's 'PR-riddled bull** written by some executive', or 'corporate talk'. If you don't speak to PR before posting something, it's 'Did you speak to PR before posting this?' There's no winning.
the whole point is deploying closer to your users, unless your customer base is the owners of those restaurants, it would be better to deploy to other (even all!) regions
Ashburn does not have "like three restaurants". The datacenter zoned part of the city has "like three restaurants". If you ever drove >5 minutes from those large shed warehouses, you'd know what you're talking about.
You are probably thinking of Sterling, Chantilly, etc.
Ashburn is literally like the Embassy Suites next to the Five Guys, the Aloft next to the taco place, Equinix, Digital Reality, Verizon, NTT, CenturyLink, etc.
I've worked on the reliability team at a place that made what they call an "edgy app" (long before fly, so we made our own stuff on baremetal) and to be fair, reliability is the actual challenge when you're talking multi-region seamless deployments of scalable web platforms. It makes or breaks that kind of product at least for the original market.
Money doesn't contribute all that much to solving it though. It's usually time to make the early mistakes, continuous improvements, etc. But AWS had several decades to solve this, whereas fly clearly hasn't, so I'm not sure I'd count them out yet.
Honestly though, Fly.io feels like it's off into the future in terms of hosting platforms. Undoubtedly they'll have to work on the reliability problems, not denying that - however, for anyone that can tolerate a bit of reliability issues, which to be honest, is more people than will readily admit it, it might just be worth the trouble for developer experience that legitimately feels like it's from the future.
It's definitely not unusably unstable, at least from my PoV, and in fact, I haven't really been hit by most of the instability personally; or at least, if I have, it has gone mostly unnoticed by me. I had an issue where I couldn't deploy for a while, but the services that were up seemed fine. (Disclaimer: I do not use the Redis/database features.)
That said, the point I'm trying to make is that it is in fact a lot nicer to use than simply YOLOing whatever onto servers. This model of PaaS is basically what App Engine should have been. It's difficult to want to go back once you've used it.
