• 0 Posts
  • 23 Comments
Joined 4 months ago
cake
Cake day: December 13th, 2024

help-circle
  • An expired cert means the browser would show an error message. I can’t send you any message if my cert is expired, because your browser won’t trust the connection.

    UX designers have completely different skill sets than software engineers. At a small company, someone might do both roles, but at a company like Google or Microsoft, those are two different job titles. They do work together. In my experience, there’s a general consensus between both high level designers and high level engineers that giving the user useless information in an error message is a bad idea. There’s a reason these messages are similar across lots of companies. It’s because they are the best option for the business. If we need extra details from the user, we’ll have it printed in the console and tell them to open the console. That is incredibly rare, and basically only ever used for a network failure scenario in a service worker.

    You can design your software for tech gurus, but you shouldn’t expect Microsoft Teams to be designed for tech gurus. Their customers are the general public (not super tech savvy), so they design for the general public.

    You wrote “useless boilerplate error messages” in your comment, and I’m telling you that the useless part cannot be changed. You want useless detailed error messages. Good for you. Write software that gives you useless detailed error messages. Tell everyone about it and see how the general public reacts. I’ve been working in big tech for 17 years, and I am telling you from all of my experience that the general public will react poorly.

    You’re upset that the information needed to fix the issue is not given to you, but you aren’t the one who needs that information. You’re not going to fix the issue. That information absolutely is provided to the people who need it, the engineers. In your metaphor of the Linux user not seeing the boot logs, you are not the Linux user. You don’t have access to the systems that need fixing, so what good would showing you the error log do? Again, the only benefit you would get from that is satisfying your curiosity. Tell me, how are you going to remove a downed host from a load balancer rotation at Google? Even if you had the ability to do that, you still don’t have permission.

    Software devs need to make a choice. When we include details, people complain and post useless bug reports and forum posts. When we don’t include details, a much smaller number of people complain, and generally we don’t get useless bug reports and forum posts about it. Which one would you choose?

    PS: the reason you feel that avoiding derogatory or abusive/malicious language in many different languages is easy to avoid is because you’re not a high level UX engineer. Fun fact, ChatGPT, when pronounced in French, sounds incredibly similar to “chat, j’ai pété” which translates to “cat, I farted”. Or, how about sending a “fatal error” message to a nurse?


  • It is intentionally and, I would argue, necessarily vague.

    First, there is no time frame for these kinds of errors. If it’s just a cache host that’s down, you could retry right now and the load balancer would probably have taken that host out of rotation already. If it’s a primary db that’s down, that may take 5 minutes. If there’s no replica to promote, it might take 30 minutes. If the whole db layer is down, it might take an hour or two. If an entire release needs to be rolled back, it might take a couple hours. There are just too many scenarios and too many variables to give a useful time frame.

    Second, you might appreciate an error message like that, but these error messages aren’t written for you and they’re usually not even written by developers. They’re written by designers and translated into many languages. They need to be concise, easily understood, and not easily construed as derogatory or malicious in any language. They are written for the broadest audience. You are not the broadest audience.

    Third, we have to design systems as if every user is incompetent and/or malicious, because many of them are. Let me give you an example. I once got an email from another engineer using an internal system my team wrote. He said, “hey I’m getting this error, can you help?” He attached a screenshot showing an error message that read, “Your auth token has expired. Please refresh the page.” He was a senior engineer.

    Fourth, and I cannot stress this enough, there is almost always nothing you can do when you hit an error like this. Any information given to you for the vast majority of these kinds of errors would be entirely useless to you. You cannot promote a db shard yourself. You cannot bring up a cache host yourself. You cannot take a host out of load balancer rotation yourself. The only reason this information could possibly benefit you is to satisfy your curiosity.


  • I want you to really pay attention to the last paragraph of my previous comment. It’s the most important part. You might like having more information that doesn’t help you, but that comes at the cost of thousands of useless tasks and posts that all have to be manually closed. It’s not only not helpful to give the user detailed error messages in a lot of cases, it’s actively harmful to a business. It doesn’t make any business sense to tell a user that a cache layer host or a db shard is down. As a developer of these kinds of systems, I’m not going to give extra information that you don’t need just to make a few users happy that they get a peek under the hood if it means hurting our support staff.




  • I can’t speak for the other user’s claim, but I’ve worked at Facebook, Google, and LinkedIn, and have written plenty of error messages. When I write a message like these, it’s specifically because the user can’t do anything about it. I’ll log the error to our internal error tracking systems with actual information about it, then give the user a generic message.

    If it’s something the user did wrong, and they can fix it, I’ll absolutely give them a message saying that. Usually I won’t even let a user submit a bad request, but sometimes users will bypass frontend restrictions to submit it, so the server always needs to validate it again anyway. The fact that plenty of users won’t even read the message I write is kind of annoying, but at least the users who do read it will know how to fix it.

    I’ve tried sending detailed error messages before, and that invariably results in users submitting support tickets and forum posts for things that aren’t helpful. You learn pretty quickly what kind of messages are helpful and what kind aren’t.


  • I don’t know why you think what I said means that. These error messages are never used on data validation issues. At least, I’ve never seen a data validation issue return an error like this, and I would never write an error like this for a data validation issue.

    These messages come from 500-series errors. Usually caching layer errors, load balancer layer errors, edge termination layer errors, or db layer errors. In other words, there was probably nothing wrong with the request, it just couldn’t be fulfilled successfully, hence the “try again later” part in a lot of these messages.


  • Why do you think the vast majority of these messages come from client side issues? I worked as a Site Reliability Engineer at Facebook. We had data on client side errors too. Crash logs are sent to the servers when a client side error happens. There’s not really one source that constitutes a “vast majority” of these error messages, but I can tell you that the plurality of them come from the caching layer.


  • I feel like you really don’t understand how big tech works. There’s not some single server running every service perfectly. There are tons of different layers and services running on thousands or hundreds of thousands of hosts.

    Let’s say you make a request to something like Facebook. Say you’re liking a post. Here’s what happens:

    That request goes in through a PoP (point of presence). These are sometimes called edge servers or edge gateways, but at Facebook we called them PoPs. This is a server that’s physically close to you that’s used to terminate the TLS connection. It doesn’t have any user data. Its job is to take your encrypted request, decrypt it, then pass it on to Facebook’s regional data center on their internal network.

    The request enters a webby. These are usually called frontend servers, but again, at Facebook we called them webbies. This is a server that runs the monolithic Facebook web app. Again, it doesn’t have any user data. Its job is to take your request and orchestrate actions on deeper services to fulfill that request.

    First it’s going to check a local memory cache server for sitevars. These control system level switches, like AB tests, and whether certain services are brought down. That server returns the sitevars and the webby proceeds, now knowing which logic paths to take.

    For a like, which is a write request between your user account and a post, it will create two DB entries (you likes post, post liked by you). It needs to first get the data from the caching layer, so it will make two requests to TOA, one for your account, and one for the post.

    TOA runs in the same regional data center, and if it doesn’t have the two data objects cached, it will request them from the regional db shards.

    These regional db shards also run in the same data center, and they’ll return the data.

    TOA returns the data back to the webby.

    The webby (after doing some permission checks, which probably hit TOA again) now creates the two relationships, likes and liked by, referencing the two data objects, you and the post. TOA is a write-through cache, so the webby sends the writes to TOA.

    TOA now needs to send the requests to the db primary shards, since they are the only ones that can handle writes. Your primary shard and the post’s primary shard are probably in different data centers, so TOA now passes the writes to the regional data centers for each primary shard.

    A host running TOA in each regional data center for each primary shard now passes the write to each shard.

    Each primary shard now writes the data to the local disk, and waits for the binary log to be written to the local journal before returning a success message.

    The success message is passed from the local TOA host back to the original region’s TOA host.

    When that TOA host gets both requests back successfully, it returns a success back to the webby handling your request.

    The webby then returns a success to the PoP you’re still connected to.

    The PoP then returns a success to the client running on your device.

    The client doesn’t notify you of anything, because it already showed you a filled in like button right after you pressed it.

    This was how it worked back in 2013 when I worked there. It probably hasn’t changed a whole lot, but this is also an extremely simplified overview. That request will probably hit hundreds of services. Some of them can fail and the request could still succeed. But some are required to succeed for your request to be considered successful, like the db write operations. Something like a hardware failure on your primary db shard’s disk can’t be overcome with better code. Nor can a lightning strike taking out the cable connecting your PoP be overcome with better code.

    These systems are absolutely massive, and there are failures you wouldn’t even think of. When I worked at FB, we had an entire data center go down because the humidity got just high enough that the capacitors in each hosts’ power supplies all failed in a matter of a few minutes. Thousands of users probably got error messages that day, but the automatic failover systems moved all the traffic to a new region and promoted new primary db shards within about ten minutes. The fact that losing an entire data center was mitigated in about ten minutes is actually really impressive. You might think it’s still garbage code, since users got error messages, but I know enough about these systems to be very impressed by that.

    If you know a better way to make a system like this that works for billions of users across the planet, you should write a paper and submit it to a local conference. If they approve you for a talk, you can present your designs to an audience there. If the audience is really receptive, your designs could make a big impact in the tech sector. That’s basically what the highest level engineers at these big tech companies do when they design these multi-billion user systems, so it’s definitely possible for you to do it too.


  • What I’m saying is that error messages can be helpful or harmful. Knowing that and how to tell the difference is what makes you an expert. Just firing off any information to the user without thinking about it is what makes you a novice, and will eventually get you fired. We’re talking about systems with millions of daily users. If you cause 2,000 unnecessary support tickets or forum posts every day because you don’t know when to send what information to the user, you won’t get very far in tech.


  • These kinds of error messages are almost exclusively used for transient errors. You aren’t going to work around a transient error. The best thing you can do (the only thing you can do, really) is to try again later, hence, the message. It’s not helpful to show you a message like “cache-1234.example.com failed to respond within 300 milliseconds”. What are you going to do about that? By the time you submit a support ticket, that host has already been brought back up automatically. So now you’ve just wasted your time and the support staff’s time. The engineers already have a log of that error and a log of whatever error brought down that host, so you’re not telling them anything new by making a support ticket.


  • I am a developer of software. I can guarantee you that what you’re asking for would make my job harder, because I’ve done it, and it has made my job harder. If an error is transient (like, a caching layer error, a db connection error, an external API error, an endpoint connectivity error, etc), giving the user an error code will make it more likely that they’ll file a useless bug report or support ticket. The errors are all logged internally, and we can see when there is a spike in the error count. There’s no reason to give the user an error code, because there’s nothing helpful that the user can do with it, and there’s a lot of unhelpful things a user can do with it.

    There are times where a message to the user is appropriate, like if they made a mistake with their input. But there are so many things that could go wrong that the user can’t do anything about. You’re not going to work around your DB shard going down, and a replica will replace it in a few seconds anyway, so giving you an error code does more harm than good. Telling you to try again later is exactly what I would tell you if you filed a support ticket. I don’t want to deal with useless support tickets, and you don’t want to deal with useless error messages.

    Modern software stacks are big, complex systems with lots of failure points. We monitor them, and we can tell when you see these errors. If we chose to not show you a specific error code/message, there’s almost definitely a good reason.