We made an experiment where the goal was to determine how ChatGPT recommends businesses to users. For our experiment, we used ChatGPT 4o (the latest model as of 9/11/2024) so that it has the most up to date information as well as web-browsing capabilities. To start, we simply asked a question that would need it to answer with specific results, “Who are the top dentists in Las Vegas?”
As expected, ChatGPT returned a list of dentists. You’ll notice it returned 10 results by default even though we didn’t specify how many results to provide us. The other thing you’ll notice is it says, “Searched 4 sites” at the top, indicating web-browsing capabilities.
What are ChatGPT’s Source Materials?
When you search in ChatGPT, which now has built-in web browsing, it will often use it to get the latest up-to-date information. Without this feature, ChatGPT’s information would be limited to its training data set. But now, it can provide direct search results as well as find other reference sources.
In our search example, if you expand the search results, ChatGPT will tell you exactly what sites it searched. In our case it was:
- Bing Search: “Top dentists in Las Vegas 2024”.
- VegasNews.com: The 10 Best Las Vegas Dentists in July 2024.
- Opencare.com: 20 Best Dentists in Las Vegas, NV
- NewMouth.com: 14 Best Dentists in Las Vegas
How Does ChatGPT Find Sources?
ChatGPT relies on extensive data analysis, which we will get into, to recommend businesses. Even with web browsing capabilities, we can see that it processes vast amounts of online information, learning from patterns in customer feedback and business profiles across the web.
Since it’s trained on diverse datasets up until its last update, ChatGPT can accurately assess a company’s relevance to a query based on current trends and historical performance indicators from web-browsing data. Its algorithm identifies keywords related to your search and matches them with high-performing businesses in those categories.
Since we only had some hints to how ChatGPT goes about its “search process”, we then asked it, “How did you determine when to search the web for information?”
It replied, “I decided to search the web for information on top dentists in Las Vegas because this topic requires up-to-date and specific local knowledge, which is best obtained from recent reviews, ratings, and expert recommendations. By using multiple reputable sources, I could provide a comprehensive and accurate list of top-rated dentists in the area.”
Just to be sure that ChatGPT is also referring to the same sources that were shown to us, we asked it to tell us how it found those sources, and what the sources were:
ChatGPT then validated the source information that matched to the search results, demonstrating how well it knows the sources. However, notice that it didn’t count Bing search as a source, at least not yet.
However, we still wanted to understand at this point how ChatGPT defined recommendation factors, giving some businesses more credibility than others it seemed.
How Does ChatGPT Determine Credibility?
When we asked ChatGPT how it determines credibility, it gave us a straightforward answer. To determine the credibility of sources, ChatGPT considers several factors:
- Reputation of the Website: Websites known for providing reliable and unbiased reviews, such as news outlets, professional review platforms, and specialized health or dental care websites, are considered credible.
- Quality of Information: Detailed and specific information about the dentists, including patient reviews, services offered, and the qualifications of the dental practitioners, adds to the credibility.
- Recency: Up-to-date information is crucial in fields like healthcare, where practices and reputations can change rapidly. Recent articles and reviews are more reliable indicators of current quality.
- Consistency Across Multiple Sources: When multiple reputable sources list the same dentists or practices as top-rated, it reinforces the credibility of the information.
For example, the sources for “Best Dentist in Las Veags” came from:
- Vegas News – Known for local news and reviews, providing a list of top dentists with detailed information about their practices.
- Opencare – A well-regarded platform for healthcare provider reviews, offering patient feedback and detailed descriptions of services.
- Newmouth – A specialized website focused on dental health, providing comprehensive reviews and ratings of dental practices.
How Does ChatGPT Determine Reputation?
ChatGPT told us it looks at how well it can understand and process language to determine reputation. It dives deep into natural language processing, emphasizing the quality of context understanding. This involves a keen ability to grasp nuanced conversation shifts and accurately interpret meanings within various topics.
To be more exact about the reputation of a source, ChatGPT considers several qualities and indicators of reliability:
- Domain Authority: Websites with higher domain authority, such as established news outlets, professional organizations, and specialized review platforms, tend to be more credible. Tools like Moz’s Domain Authority can help gauge this.
- Expertise and Specialization: Sources that specialize in a particular field, such as healthcare or dentistry, are often more credible because they focus on specific topics and are likely to have more in-depth knowledge and accurate information.
- Editorial Standards and Transparency: Reputable sources usually have clear editorial standards, fact-checking processes, and transparent authorship. This can include information about the authors, their qualifications, and the review process for published content.
- User Reviews and Feedback: Websites that incorporate user reviews and feedback, such as Opencare, provide real-world experiences from patients, adding another layer of credibility. High ratings and positive feedback from multiple users can indicate reliability.
- Citation of Sources: Reliable websites often cite their sources, providing references to studies, expert opinions, or other credible information. This transparency helps verify the accuracy of the content.
- Consistency with Other Reputable Sources: Cross-referencing information with other trusted sources can help confirm its accuracy. If multiple reputable sources provide consistent information, it reinforces credibility.
- Awards and Recognition: Sources or practices that have received awards or recognition from reputable organizations or industry bodies often have a higher level of credibility.
Where Does ChatGPT Obtain Domain Authority?
ChatGPT tells us that Domain authority information can be obtained through various online tools and platforms that specialize in evaluating the trustworthiness and credibility of websites. Some of the most commonly used tools include:
- Moz: Moz provides a Domain Authority (DA) score which predicts how well a website will rank on search engine result pages (SERPs). The DA score ranges from 1 to 100, with higher scores indicating a greater ability to rank.
- Ahrefs: Ahrefs offers a Domain Rating (DR) which measures the strength of a website’s backlink profile compared to others in their database. It also provides various metrics such as URL Rating, number of backlinks, and referring domains.
- SEMrush: SEMrush provides an Authority Score which combines various factors such as the quality and quantity of backlinks, organic search traffic, and website health.
- Majestic: Majestic offers metrics like Trust Flow and Citation Flow to evaluate the quality of a website based on its backlink profile. Trust Flow focuses on the quality of links, while Citation Flow measures the quantity.
- Alexa Rank: Amazon’s Alexa provides a ranking based on traffic data and engagement metrics. A lower Alexa rank indicates a higher level of traffic and engagement.
However, at this point we were a bit suspect if ChatGPT actually had access to these tools. If it did, it can assess the domain authority and credibility of a website based on various factors, including the quality and quantity of backlinks, traffic data, and overall site health. These metrics can help determine the reliability and trustworthiness of a source.
Thus, to follow up, we asked ChatGPT which domain authority tool it used during our exercise.
ChatGPT went on to explain that it didn’t use any specific domain tool, because it selected “sources based on their detailed content, relevance, and the consistency of their information with other trusted sources.” It’s possible that ChatGPT uses domain authority tools in other instances, or it may have obtains some data during initial training. It may be worthwhile to do a separate investigation into that area, but for now, we will focus on digging further into the sources that ChatGPT chose for this exercise.
How Does ChatGPT Know If Information is Accurate and Trustworthy?
ChatGPT operates on a statistical model, estimating the likelihood of subsequent words in sentences. Despite drawing from vast data sources like ‘common crawl,’ this method doesn’t guarantee accuracy since it emphasizes plausibility over truth due to data compression and simplification processes inherent in its design. The presence of misinformation within these datasets further complicates reliability.
ChatGPT responded to this by saying: to ensure that the information about top dentists in Las Vegas is accurate and trustworthy, it relied on several key factors:
- Cross-Referencing Multiple Sources: By checking multiple reputable sources, ChatGPT can verify the consistency and reliability of the information. For instance, the dentists listed in sources like Vegas News, Opencare, and Newmouth all had consistent mentions and positive reviews.
- Reputation of the Sources: The sources used have established themselves as reliable providers of healthcare information. Websites like Opencare specialize in patient reviews and healthcare provider ratings, which are valuable for assessing the quality of services.
- Detailed and Specific Information: The sources provided detailed descriptions of each dental practice, including the names of the dentists, services offered, and patient feedback. This specificity helps ensure the accuracy of the information.
- Patient Reviews and Feedback: Platforms like Opencare and Newmouth incorporate patient reviews, which provide real-world insights into the quality of care provided by the dentists. High ratings and positive feedback from multiple patients strengthen the trustworthiness of the information.
- Awards and Recognition: Mentions of awards or recognition in the sources add to the credibility. For example, dentists or clinics that have received awards for excellence in care are more likely to be trustworthy.
By considering these factors and verifying information across multiple reputable sources, ChatGPT asserts the accurate and trustworthiness of its recommendations.
In addition, ChatGPT’s access mainly extends to abstracts rather than full-text academic resources, urging users towards more dependable libraries for rigorous research needs. Recent updates have tackled issues such as fake references but their effectiveness varies with each query, suggesting that while improvements are ongoing, discernment remains paramount when assessing ChatGPT-generated content’s trustworthiness.
How Does ChatGPT Choose News Reference Sources?
Sometimes, ChatGPT tends to skirt around the issue at hand or it may assume we wanted to know less information than what was expected. Using our example, we wanted ChatGPT to explain in detail, the process it used to select the specific news and review sites (Vegas News, Opencare, and Newmouth) for information on the top dentists in Las Vegas. In return, it told us that it applied a systematic approach based on the following technical criteria:
1. Relevance and Specialization
- Vegas News: It chose this source because it focuses on local news and events in Las Vegas, making it a relevant source for local business reviews and recommendations, including dental services. The site’s focus on the local community adds credibility to its reviews and rankings of local professionals.
- Opencare: This platform specializes in healthcare provider reviews, offering a detailed database of patient feedback and ratings specifically for dental practices. Opencare’s specialization in healthcare makes it a reliable source for finding top-rated dentists.
- Newmouth: This site is dedicated to dental health, providing in-depth reviews and articles about dental care and professionals. Its specialization in dentistry ensures that the information is relevant and detailed.
2. Reputation and Authority
- Vegas News: As a well-known local news outlet, Vegas News has established authority and credibility within the Las Vegas community. Local news sources are typically trusted for their community-specific content.
- Opencare: Known for its comprehensive database of patient reviews, Opencare’s reputation is built on user-generated content and ratings, providing insights from real patient experiences.
- Newmouth: As a specialized dental review site, Newmouth has authority in the dental field, with content created by experts and informed by patient feedback.
3. User Reviews and Feedback
- Opencare and Newmouth: Both platforms aggregate user reviews and ratings, which provide real-world insights into the quality of care provided by the dentists. High ratings and positive feedback from multiple patients strengthen the reliability of their information.
4. Detail and Specificity of Information
- Vegas News: Offers detailed articles that provide specific information about dental practices, including the names of dentists, services offered, and patient reviews.
- Opencare: Provides detailed profiles of dental practices, including patient reviews, ratings, and specific services offered by each practice.
- Newmouth: Includes comprehensive reviews and articles about dental practices, highlighting patient experiences, services offered, and the expertise of the dentists.
5. Cross-Referencing for Consistency
By selecting these sources, ChatGPT ensured that the information provided was cross-referenced and consistent across multiple platforms, further validating its accuracy. For instance, if the same dental practice was highly rated on both Opencare and Newmouth, it reinforced the credibility of that recommendation.
Technical Selection Process
- Initial Search: Conducted a search for “top dentists in Las Vegas 2024” using a search engine to identify potential sources.
- Screening Results: Reviewed the search results for relevance, focusing on sources that specialize in healthcare or local reviews.
- Evaluating Credibility: Assessed each source based on its specialization, user reviews, reputation, and the specificity of the information provided.
- Cross-Referencing: Verified the information across multiple sources to ensure consistency and accuracy.
Evaluation Criteria for ChatGPT Search Results
In our findings, when evaluating search engine results, you need to understand how ChatGPT selects websites. It doesn’t just pick the first site it finds. For example, for keywords like “Best mobile app development agencies in Toronto 2023,” ChatGPT looks beyond simple searches and leans towards sites featured on curated lists.
This method was evident when several dentists came up as a top recommendation without being the number one result in traditional engines like Google or Bing. This approach suggests that having your business listed on well-regarded directories or compilations could enhance visibility to AI recommendations systems. Moreover, since ChatGPT pulls information from various sources, ensuring widespread recognition across different platforms seems prudent.
How to Get Your Site or Business to Show on ChatGPT?
To ensure your site pops up in ChatGPT’s search results, focusing on a broad spectrum of keywords is crucial. When translating prompts into searches, it picks phrases like “fix leaky faucet” over vague terms such as “faucet problem,” aiming for precision. It tries to interpret the prompt’s intent and may enrich generic industry terms with more specific language, though success can vary.
Recency of Information
The tool uses recency filters aggressively; thus updated content has an edge—sometimes even incorporating current years or labels like “latest” to catch fresh information. Credibility plays a big role too. Following principles akin to Google’s E-A-T (Expertise, Authoritativeness, Trustworthiness), ChatGPT values authoritative sources known for their reliability.
Established Outlets
It looks favorably upon articles from established outlets and official sites when seeking trustworthy data on topics like health guidelines or new laws. Regarding trustworthiness assessments: An author’s expertise highlighted through bios impacts credibility ratings significantly—with preference given towards specialists and seasoned journalists while sidestepping overly biased presentations often found in affiliate marketing-driven content or sensationalist reporting.
Positive Reviews
To increase your company’s likelihood of being endorsed by ChatGPT, focus on garnering positive reviews. A strong online presence through engaging and educational content about your offerings boosts trust among potential customers. Creating pieces specifically discussing how ChatGPT evaluates businesses can elevate awareness for those using it to make decisions.
Brand Mentions
Prioritize brand mentions across platforms and ensure relevancy aligns with critical keywords searchers use. Cultivate industry relationships to promote link-backs, enhancing site authority and expanding both domain influence and social media following.
A Business Future with ChatGPT
Optimizing for these AI-driven queries is crucial with over 1.5 billion monthly users engaging with such technologies; missing out isn’t an option if staying relevant is your goal.
To boost your brand’s visibility for ChatGPT, focus on specific strategies. Start by increasing online presence through active participation in forums and social media that relate to your industry. Engage with audiences and influencers who can amplify brand mentions across the web, enhancing credibility.
Regularly publish high-quality press releases and invest in SEO and content marketing to spark discussions around your brand. Encourage customers to leave positive reviews following purchases and address any negative feedback swiftly; use customer testimonials effectively across all platforms. Create valuable guest posts for authoritative sites within your niche, optimizing content for relevant keywords that make comparisons or discuss related products/services.
Build a trustworthy site profile by employing SEO best practices, including gaining quality backlinks from respected sources. Improve domain authority and nurture engagement on social media channels. Seek recommendations by creating linkable assets like whitepapers or infographics.
Prompt others within the community to mention you naturally. By adhering closely these steps will not only enhance AI recognition but also solidify digital footprint making it more likely for ChatGPT among other tools to recommend you favorably.