How to Verify Domain Ownership in Scraping Camel via DNS#
When creating a new project (website) in the Scraping Camel extension, you must prove your relationship to the domain. This measure prevents the tool from being misused for scraping third-party websites or competitors. One of the most straightforward methods, especially for marketing agencies, is verification using a DNS TXT record.
A DNS TXT record is a specific type of data record in the Domain Name System (DNS) that allows you to associate any text string with your domain. While other record types (such as A or MX) point a domain to a specific IP address or mail server, a TXT record is used primarily to pass information to external services.
In the context of Scraping Camel, it works as a “text note” that proves domain ownership. You insert a unique verification code, generated by the extension, into your hosting admin, and Scraping Camel then verifies that this code is actually present in the DNS records. This confirms that you have access to configure the given website.
How to complete verification in Scraping Camel#
1. Getting the verification code in Scraping Camel#
The first step is to create a new project in the extension.
- In the Websites section, click the New website button.
- In the wizard, enter the name and URL of your website.
- In the domain verification step, select the DNS record method.
- The extension will generate a unique text string (value) — copy it.
2. Setting up the DNS record with your domain provider#
You now need to insert this code into your domain settings. This is done in the admin panel of your web hosting provider or domain registrar.
- Log in to your hosting admin. The admin interface varies by hosting provider, but look for a section with a name similar to Domain Management or DNS Settings.
- Create a new DNS record with the following parameters:
- Record type: Select TXT.
- Name: Enter exactly the subdomain you used when creating the project in Scraping Camel.
- Value: Paste the unique code you copied from the wizard in the extension into this field.
⚠️ Important: The Name field must exactly match how you entered the domain in the extension. If you entered
www.yourdomain.comin the wizard, enter www here too. If you entered the domain without www (i.e. yourdomain.com), leave the field empty or enter @ — depending on your hosting interface. Mixing up these variants is the most common cause of failed verification.
3. Completing the verification#
After saving the new DNS record, you need to wait for it to propagate across the network.
- DNS changes do not take effect immediately. Hosting providers typically cache these records, so the process can take 15 to 60 minutes (or sometimes longer).
- You can check whether the record is already active using online tools, for example Google Dig, where you enter the domain (including www if applicable) and the tool returns the DNS records it finds. Linux users can verify this by running dig txt your-domain.com in the terminal.
- Once the record is visible, return to Scraping Camel and continue through the wizard to the next step. The system will verify the validity of the record on its own.
FAQ#
Why do I need to verify domain ownership?#
Scraping Camel is designed exclusively for processing your own websites or those of your clients. Domain verification prevents the tool from being misused for scraping third-party or competitor websites, and ensures that only an authorised person is working with the data.
What is a DNS TXT record?#
It is a special type of record in the Domain Name System that allows you to associate any text string with a domain. Unlike other records (such as A or MX), it is not used to redirect traffic but to pass information to external services — in this case as proof of domain ownership.
Why choose the DNS TXT record method?#
This method is permanent and independent of any changes to the website’s code. It is particularly suitable when you do not want to modify the source code of pages, or when you manage the domain directly with your registrar and have DNS access.
Where do I set up the DNS record?#
In the admin panel of your web hosting provider or domain registrar. The exact location varies by provider — you are typically looking for a section called Domain Management or DNS Settings.
What parameters do I need to fill in when creating the record?#
Set the record type to TXT. In the Name field, enter exactly the subdomain you specified when creating the project in Scraping Camel. In the Value field, paste the unique code generated by the extension.
How should I fill in the Name field — with www or without?#
It depends on how you entered the domain in the Scraping Camel wizard. If you entered www.yourdomain.com, enter www. If you entered the domain without www, leave the field empty or enter @ — depending on your hosting interface. Mixing up these variants is the most common cause of failed verification.
How long does it take for a DNS record to propagate?#
Usually 15 to 60 minutes, but propagation can take longer depending on your hosting configuration.
How do I know the DNS record is already active?#
You can verify this using the Google Dig online tool — enter the domain including www if applicable, and the tool will display the DNS records it finds.
What should I do if verification in Scraping Camel is still not working?#
Check that the Name field in the DNS record exactly matches the subdomain entered in the wizard (www vs. without www). If the record is visible in Google Dig but Scraping Camel still cannot find it, try running the verification again — propagation to all servers can take a while.