When to Use It
- Clean campaign lists that may have duplicate entries
- Remove repeated URLs from sitemap extraction
- Deduplicate client lists from multiple sources
- Ensure unique keywords before processing
Inputs
Outputs
Credit Cost
Free to use - no credits required.Real-World Examples
Clean Campaign Data:How It Works
The node compares items and keeps only the first occurrence of each unique value: Example Process:- Text comparison is case-sensitive: “Apple” ≠ “apple”
- Numbers are compared by value: 123 = 123.0
- Empty values are treated as duplicates if multiple exist
Tips
Data Quality:- Always use this before loops to avoid processing the same item multiple times
- Helps reduce API calls and processing time
- Essential when combining data from multiple sources
- Prevents duplicate entries in final outputs
- Reduces workflow execution time by eliminating redundant processing
- Especially important for large lists with many duplicates
- Remember that “Campaign A” and “campaign a” are different items
- Consider standardizing text case before deduplication if needed
FAQ
Does this preserve the original order of items?
Does this preserve the original order of items?
Yes, the node keeps items in their original order. It removes duplicates but maintains the sequence of the first occurrence of each unique item.
How does it handle case sensitivity?
How does it handle case sensitivity?
The comparison is case-sensitive, so “Apple” and “apple” are treated as different items. If you need case-insensitive deduplication, standardize the text case first.
What happens if my entire list contains duplicates?
What happens if my entire list contains duplicates?
You’ll get a much shorter list with only unique items. For example, [“A”, “A”, “A”, “B”, “B”] becomes [“A”, “B”].
Can this remove duplicates from complex data like campaign objects?
Can this remove duplicates from complex data like campaign objects?
Yes, but it compares the entire data structure. Two campaign objects are only considered duplicates if every field matches exactly. For partial matching, use other filtering methods.
How do I know how many duplicates were removed?
How do I know how many duplicates were removed?
Use Count List Items before and after Remove Duplicates, then subtract to see how many duplicates were found. This helps assess data quality.
Does this work with numbers and mixed data types?
Does this work with numbers and mixed data types?
Yes, it works with any data type. Numbers are compared by value (123 equals 123), text by exact match, and mixed lists handle each type appropriately.