Optimizing Claygent for Duplicate Company Data in LMS Scraping

I’m using Claygent to scrape LMS information for companies in my dataset, but I’m running into an issue with multiple rows for the same company. To avoid using extra credits, I’d like Claygent to skip re-running for a company once the LMS has been identified in any row and instead automatically populate the LMS value across all rows with that company name. Could you help me set this up or point me to any resources for managing data across rows with duplicate company names? https://app.clay.com/workspaces/242700/tables/t_rGuGx7jycdu6/views/gv_eT78pCTsUUe4

17 comments

Muhammad S.
·9 months ago·
One workaround: 1- Create two tables. 2- Do auto-dedupe on the company name column to remove duplicate companies from one table. 3- Once all the duplicate companies are removed, run claygent and map the required data in the columns. 4- In the 2nd table - Do a lookup record to pull the data from the first table where you enriched companies. Not sure if there if this is the best workaround.
Drew C.
·9 months ago·
If you could give me a simple step by step process. I have an excel spreadsheet with 1465 contacts belonging to 60ish companies. All I want is the LMS value (see screenshot) to populate in the LMS Value Addition field in this table: https://app.clay.com/workspaces/242700/tables/t_rGuGx7jycdu6/views/gv_eT78pCTsUUe4
Drew C.
·9 months ago·
Muhammad S.
·9 months ago·
1- Create a 2nd table 2- Copy all the companies from the 1st table(original table containing the companies) to the 2nd table. 3- Dedupe on the company domain to remove duplicate entries in the 2nd table - Now you have a 2nd table with all the unique companies in it. 4- Run the claygent prompt on the 2nd table(containing unique companies) 5- Use lookup single row in other table enrichment in the 1st table and lookup by company domain in the 2nd table 6- Map the results returned by the lookup enrichment. Does this make sense If you follow this step by step should be easy
Drew C.
·9 months ago·
Thanks Muhammad - I'll give this a whirl tomorrow morning. Might have a chat with you later this week about outsourcing projects.
1
Drew C.
·9 months ago·
Shall I book on to your calendar?
Muhammad S.
·9 months ago·
sure
1
Bruno R.
·9 months ago·
Hey Drew, thanks for reaching out! If you're looking to cache your prompts results, I suggest that you try this lookup + write to table setup which is particularly useful for streaming setups. Here's how you can build this out: https://www.loom.com/share/c90b34abc6f04f848be32fdc861a0d86?sid=5da240cc-c774-4cec-be55-1434f9191a94 I hope this was helpful. If there's anything else I can assist you with, please let me know!
Drew C.
·9 months ago·
Tried these solutions. Not getting it to work so far. Where I'm up to. I've created two tables. One with company names which is labelled Step 1: https://app.clay.com/workspaces/242700/workbooks/wb_g9wQXp4wSwzi/tables/t_j9miHy9wGnqT/views/gv_T3zWDek2dbTz In row 1, in table 1, you can see from screenshot, we've identified the LMS in the column labelled: "Learning Platforms Used Response". However, as per screenshot in table 2: https://app.clay.com/workspaces/242700/tables/t_rGuGx7jycdu6/views/gv_eT78pCTsUUe4 we are getting no record identified. How do I look up the record and populate all the companies with this value?
Muhammad S.
·9 months ago·
This should be company domain:
Muhammad S.
·9 months ago·
This is the value you are trying to find in the other table so it has to be something that is available/common in both tables.
Drew C.
·9 months ago·
hi mate, like attached?
Muhammad S.
·9 months ago·
yes sir
Drew C.
·9 months ago·
thought so. Thanks mate. It's not working. See what Clay support says.
Drew C.
·9 months ago·
Bruno R. When you're back can you take a look at the tables. Please see why the value is not being pulled through Here are the two tables: https://app.clay.com/workspaces/242700/workbooks/wb_g9wQXp4wSwzi/tables/t_j9miHy9wGnqT/views/gv_T3zWDek2dbTz https://app.clay.com/workspaces/242700/tables/t_rGuGx7jycdu6/views/gv_eT78pCTsUUe4
Tanvi R.
·9 months ago·
Hey Drew, it looks like the values are not being pulled with the Lookup enrichment because in the Step 2 table, the company domain starts with http:// but in the Step 1 table the domain starts with https://. Thus, the lookup views both of these domains as different values which is why none of the rows are returning a record found. To resolve this, I've *normalized the domain*s in your step 2 table using the Normalize domain enrichment to remove the https:// and made this normalized url column the row value to look up in the other table. I also used the contains operator in your setup as this works better when comparing text/string values. It's working successfully now - let me know if this makes sense and if you have any other questions!
1
Channeled
APP
·9 months ago·
Hi Drew C.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:

Optimizing Claygent for Duplicate Company Data in LMS Scraping | Clay