Microsoft Says Revamped Bing is ‘Work in Progress’ After Report Claims Mistakes in Chatbot’s Responses

Just over a week after Microsoft unveiled its new Bing search engine powered by the technology behind the ChatGPT artificial-intelligence chatbot, The Wall Street Journal reported that early testers were calling out mistakes and disturbing responses generated by the technology.

Microsoft said that the search engine is still a work in progress, describing the past week as a learning experience that is helping it test and improve the new Bing. So far, only a select set of people have been given access to it. The company said in a blog post late Wednesday that the Bing upgrade is “not a replacement or substitute for the search engine, rather a tool to better understand and make sense of the world.”

Microsoft unveiled the upgraded Bing during an event last week at its Redmond, Washington, headquarters, according to the Wall Street Journal.

The company said the change enables a new kind of search in which people will pose questions to the search engine in natural language and Bing will generate direct answers and suggestions, as opposed to pointing users toward different websites.

The new Bing is going to “completely change what people can expect from search,” Microsoft chief executive, Satya Nadella, told The Wall Street Journal ahead of the launch.

Some parts of the demonstration were problematic: Microsoft was showing how Bing can generate and compare tables on public companies’ earnings results with regular language prompts, but the information Bing displayed contained mistakes, according to WSJ.

In the days that followed, people began sharing their experiences online, with many pointing out errors and confusing responses. When one user asked Bing to write a news article about the Super Bowl “that just happened,” Bing gave the details of last year’s championship football game.

On social media, many early users posted screenshots of long interactions they had with the new Bing. In some cases, the search engine’s comments seem to show a dark side of the technology where it seems to become unhinged, expressing anger, obsession and even threats.

In its blog, Microsoft said the feedback on the new Bing so far has been mostly positive, with 71 percent of users giving it the “thumbs-up.” The company also discussed the criticism and concerns.

“Some of you have encountered and reported technical issues or bugs with the new Bing, such as slow loading, broken links, or incorrect formatting,” the company said. “Many of these issues have been addressed with our daily releases and even more will be addressed with our larger releases each week.”

Microsoft said it discovered that Bing starts coming up with strange answers following chat sessions of 15 or more questions and that it can become repetitive or respond in ways that don’t align with its designed tone.

The company said it was trying to train the technology to be more reliable at finding the latest sports scores and financial data. It is also considering adding a toggle switch, which would allow users to decide whether they want Bing to be more or less creative with its responses.

OpenAI also chimed in on the growing negative attention on the technology, WSJ said. In a blog post on Thursday it outlined how it takes time to train and refine ChatGPT and having people use it is the way to find and fix its biases and other unwanted outcomes.

“Many are rightly worried about biases in the design and impact of AI systems,” the blog said. “We are committed to robustly addressing this issue and being transparent about both our intentions and our progress.”

The Wall Street Journal said Microsoft’s quick response to user feedback reflects the importance it sees in people’s reactions to the budding technology as it looks to capitalise on the breakout success of ChatGPT. The company is aiming to use the technology to push back against Alphabet’s dominance in search through its Google unit.