Bạn đang xem bài viết 7 Ways To Find And Remove Duplicate Values In Microsoft Excel được cập nhật mới nhất trên website Hoisinhvienqnam.edu.vn. Hy vọng những thông tin mà chúng tôi đã chia sẻ là hữu ích với bạn. Nếu nội dung hay, ý nghĩa bạn hãy chia sẻ với bạn bè của mình và luôn theo dõi, ủng hộ chúng tôi để cập nhật những thông tin mới nhất.
Duplicate values in your data can be a big problem! It can lead to substantial errors and over estimate your results.
But finding and removing them from your data is actually quite easy in Excel.
In this tutorial, we are going to look at 7 different methods to locate and remove duplicate values from your data.
Video Tutorial What Is A Duplicate Value?Duplicate values happen when the same value or set of values appear in your data.
In the above example, there is a simple set of data with 3 columns for the Make, Model and Year for a list of cars.
The first image highlights all the duplicates based only on the Make of the car.
The second image highlights all the duplicates based on the Make and Model of the car. This results in one less duplicate.
The second image highlights all the duplicates based on all columns in the table. This results in even less values being considered duplicates.
The results from duplicates based on a single column vs the entire table can be very different. You should always be aware which version you want and what Excel is doing.
Find And Remove Duplicate Values With The Remove Duplicates CommandRemoving duplicate values in data is a very common task. It’s so common, there’s a dedicated command to do it in the ribbon.
You then need to tell Excel if the data contains column headers in the first row. If this is checked, then the first row of data will be excluded when finding and removing duplicate values.
You can then select which columns to use to determine duplicates. There are also handy Select All and Unselect All buttons above you can use if you’ve got a long list of columns in your data.
This command will alter your data so it’s best to perform the command on a copy of your data to retain the original data intact.
Find And Remove Duplicate Values With Advanced Filters
You can choose to either to Filter the list in place or Copy to another location. Filtering the list in place will hide rows containing any duplicates while copying to another location will create a copy of the data.
Excel will guess the range of data, but you can adjust it in the List range. The Criteria range can be left blank and the Copy to field will need to be filled if the Copy to another location option was chosen.
Check the box for Unique records only.
Press OK and you will eliminate the duplicate values.
Find And Remove Duplicate Values With A Pivot TablePivot tables are just for analyzing your data, right?
You can actually use them to remove duplicate data as well!
You won’t actually be removing duplicate values from your data with this method, you will be using a pivot table to display only the unique values from the data set.
First, create a pivot table based on your data. Select a cell inside your data or the entire range of data ➜ go to the Insert tab ➜ select PivotTable ➜ press OK in the Create PivotTable dialog box.
Select the Show in Tabular Form option.
Select the Repeat All Item Labels option.
Pivot tables only list unique values for items in the Rows area, so this pivot table will automatically remove any duplicates in your data.
Find And Remove Duplicate Values With Power QueryPower Query is all about data transformation, so you can be sure it has the ability to find and remove duplicate values.
Remove Duplicates Based On One Or More ColumnsWith Power Query, you can remove duplicates based on one or more columns in the table.
You need to select which columns to remove duplicates based on. You can hold Ctrl to select multiple columns.
You can also access this command from the Home tab ➜ Remove Rows ➜ Remove Duplicates.
= Table.Distinct(#"Previous Step", {"Make", "Model"})If you look at the formula that’s created, it is using the Table.Distinct function with the second parameter referencing which columns to use.
Remove Duplicates Based On The Entire TableTo remove duplicates based on the entire table, you could select all the columns in the table then remove duplicates. But there is a faster method that doesn’t require selecting all the columns.
= Table.Distinct(#"Previous Step")If you look at the formula that’s created, it uses the same Table.Distinct function with no second parameter. Without the second parameter, the function will act on the whole table.
Keep Duplicates Based On A Single Column Or On The Entire TableIn Power Query, there are also commands for keeping duplicates for selected columns or for the entire table.
Follow the same steps as removing duplicates, but use the Keep Rows ➜ Keep Duplicates command instead. This will show you all the data that has a duplicate value.
Find And Remove Duplicate Values Using A FormulaYou can use a formula to help you find duplicate values in your data.
= [@Make] & [@Model] & [@Year]The above formula will concatenate all three columns into a single column. It uses the ampersand operator to join each column.
= TEXTJOIN("", FALSE , CarList[@[Make]:[Year]])If you have a long list of columns to combine, you can use the above formula instead. This way you can simply reference all the columns as a single range.
= COUNTIFS($E$3:E3, E3)Copy the above formula down the column and it will count the number of times the current value appears in the list of values above.
If the count is 1 then it’s the first time the value is appearing in the data and you will keep this in your set of unique values. If the count is 2 or more then the value has already appeared in the data and it is a duplicate value which can be removed.
Add filters to your data list.
Now you can filter on the Count column. Filtering on 1 will produce all the unique values and remove any duplicates.
You can then select the visible cells from the resulting filter to copy and paste elsewhere. Use the keyboard shortcut Alt + ; to select only the visible cells.
Find And Remove Duplicate Values With Conditional FormattingWith conditional formatting, there’s a way to highlight duplicate values in your data.
Just like the formula method, you need to add a helper column that combines the data from columns. The conditional formatting doesn’t work with data across rows, so you’ll need this combined column if you want to detect duplicates based on more than one column.
You can select to either highlight Duplicate or Unique values.
You can also choose from a selection of predefined cell formats to highlight the values or create your own custom format.
Select Filter by Color in the menu.
Filter on the color used in the conditional formatting to select duplicate values or filter on No Fill to select unique values.
You can then select just the visible cells with the keyboard shortcut Alt + ;.
Find And Remove Duplicate Values Using VBAThere is a built in command in VBA for removing duplicates within list objects.
Sub RemoveDuplicates() Dim DuplicateValues As Range Set DuplicateValues = ActiveSheet.ListObjects("CarList").Range DuplicateValues.RemoveDuplicates Columns:=Array(1, 2, 3), Header:=xlYes End SubThe above procedure will remove duplicates from an Excel table named CarList.
Columns:=Array(1, 2, 3)The above part of the procedure will set which columns to base duplicate detection on. In this case it will be on the entire table since all three columns are listed.
Header:=xlYesThe above part of the procedure tells Excel the first row in our list contains column headings.
You will want to create a copy of your data before running this VBA code, as it can’t be undone after the code runs.
ConclusionsDuplicate values in your data can be a big obstacle to a clean data set.
Thankfully, there are many options in Excel to easily remove those pesky duplicate values.
So, what’s your go to method to remove duplicates?
How To Highlight Duplicate Values In Excel
Image: Aajan Getty Images/iStockphoto
The article, How to highlight unique values in Excel, shows two easy ways to apply conditional formatting to unique values or the row that contains a unique value. In this article, we’ll do the same thing with duplicate values. We’ll first review the easy built-in rule that formats duplicate values. Then, we’ll apply a conditional format rule that highlights the entire record.
SEE: 69 Excel tips every user should master (TechRepublic)
I’m using Microsoft 365 on a Windows 10 64-bit system, but you can use an earlier version. You can work with your own data or download the demonstration .xlsx file. The browser supports conditional formatting; however, you can’t use the browser to implement a formula rule.
How to highlight individual values in ExcelFigure A Figure B Figure C
The simple data set shown in Figure A repeats a few values in column D: 1, 2, and 6. They’re easy to discern visually, but that won’t always be the case. Let’s use the built-in rule to highlight them:
Select the values you want to format; in this case that’s D3:D16.
From the dropdown, choose Highlight Cells Rules, and then choose Duplicate Values from the resulting submenu (Figure A).
Choose a preset format from the dropdown to the right (Figure B).
A built-in rule is easy to implement and might be adequate. When it isn’t, you might have to turn to a formulaic rule.
How to highlight rows in ExcelFor better or worse, you can’t use a built-in rule to highlight the entire row when column D contains a duplicate value. For that, we’ll need a formula that relies on a COUNTIFS() in the form
Select the data range, B3:E16–you want to highlight the entire row. If you use a Table, Excel will update range as you add and delete records.
Figure D
COUNTIF( range, criteria)
where range identifies the entire data set (record) and criteria specifies the condition, which can be a cell reference, a value, or even an expression. Let’s try that now:
The COUNTIFS() function itself counts the number of times a value occurs in column D. If that value is greater than one, meaning the value occurs more than once, the function returns True and the format is applied. When the count is 1 or less, the function returns False, and nothing happens.
You have two conditional formatting rules to work with now. One is built-in and highlights individual values. The other is a formula and highlights the entire record.
Also seeHow To Identify Duplicates In Excel: Find, Highlight, Count, Filter, Etc.
The tutorial explains how to search for duplicates in Excel. You will learn a few formulas to identify duplicate values or find duplicate rows with or without first occurrences. You will also learn how to count instances of each duplicate record individually and find the total number of dupes in a column, how to filter out duplicates, and more.
And today, I’d like to share a few quick and effective methods to identify duplicates in a single list. These solutions work in all versions of Excel 365, Excel 2023, Excel 2023, Excel 2013 and lower.
How to identify duplicates in ExcelThe easiest way to detect duplicates in Excel is using the COUNTIF function. Depending on whether you want to find duplicate values with or without first occurrences, there’s going to be a slight variation in the formula as shown in the following examples.
How to find duplicate records including 1st occurrencesSupposing you have a list of items in column A that you want to check for duplicates. These can be invoices, product Id’s, names or any other data.
Here’s a formula to find duplicates in Excel including first occurrences (where A2 is the topmost cell):
Input the above formula in B2, then select B2 and drag the fill handle to copy the formula down to other cells:
As you can see in the screenshot above, the formula returns TRUE for duplicate values and FALSE for unique values.
Note. If you need to find duplicates in a range of cells rather than in an entire column, remember to lock that range with the $ sign. For example, to search for duplicates in cells A2:A8, use this formula:
For a duplicate formula to return something more meaningful than the Boolean values of TRUE and FALSE, enclose it in the IF function and type any labels you want for duplicate and unique values:
In case, you want an Excel formula to find duplicates only, replace “Unique” with an empty string (“”) like this:
The formula will return “Duplicates” for duplicate records, and a blank cell for unique records:
How to search for duplicates in Excel without 1st occurrencesIn case you plan to filter or remove duplicates after finding them, using the above formula is not safe because it marks all identical records as duplicates. And if you want to keep the unique values in your list, then you cannot delete all duplicate records, you need to only delete the 2nd and all subsequent instances.
So, let’s modify our Excel duplicate formula by using absolute and relative cell references where appropriate:
As you can see in the following screenshot, this formula does not identity the first occurrence of “Apples” as duplicate:
How to find case-sensitive duplicates in ExcelIn situations when you need to identify exact duplicates including the text case, use this generic array formula (entered by pressing Ctrl + Shift + Enter):
IF( SUM(( –EXACT(range, uppermost _cell)))<=1, “”, “Duplicate”)
At the heart of the formula, you use the EXACT function to compare the target cell with each cell in the specified range exactly. The result of this operation is an array of TRUE (match) and FALSE (not match), which is coerced to an array of 1’s and 0’s by the unary operator (–). After that, the SUM function adds up the numbers, and if the sum is greater than 1, the IF function reports a “Duplicate”.
For our sample dataset, the formula goes as follows:
=IF(SUM((--EXACT($A$2:$A$8,A2)))<=1,"","Duplicate")
As shown in the screenshot below, it treats lowercase and uppercase as different characters (APPLES is not identified as a duplicate):
How to find duplicate rows in ExcelIf your aim is to dedupe a table consisting of several columns, then you need a formula that can check each column and identify only absolute duplicate rows, i.e. rows that have completely equal values in all columns.
Let’s consider the following example. Supposing, you have order numbers in column A, dates in column B, and ordered items in column C, and you want to find duplicate rows with the same order number, date and item. For this, we are going to create a duplicate formula based on the COUNTIFS function that allows checking multiple criteria at a time:
To search for duplicate rows with 1st occurrences, use this formula:
The following screenshot demonstrates that the formula really locates only the rows that have identical values in all 3 columns. For example, row 8 has the same order number and date as rows 2 and 5, but a different item in column C, and therefore it is not marked as duplicate row:
To show duplicate rows without 1st occurrences, make a little adjustment to the above formula:
How to count duplicates in ExcelIf you want to know the exact number of identical records in your Excel sheet, use one of the following formulas to count duplicates.
Count instances of each duplicate record individuallyWhen you have a column with duplicated values, you may often need to know how many duplicates are there for each of those values.
=COUNTIF($A$2:$A$8, $A2)
As demonstrated in the following screenshot, the formula counts the occurrences of each item: “Apples” occurs 3 times, “Green bananas” – 2 times, “Bananas” and “Oranges” only once.
If you want to identify 1st, 2nd, 3rd, etc. occurrences of each item, use the following formula:
=COUNTIF($A$2:$A2, $A2)
In a similar manner, you can count the occurrences of duplicated rows. The only difference is that you will need to use the COUNTIFS function instead of COUNTIF. For example:
=COUNTIFS($A$2:$A$8, $A2, $B$2:$B$8, $B2)
Once the duplicate values are counted, you can hide unique values and only view duplicates, or vice versa. To do this, apply Excel’s auto-filter as demonstrated in the following example: How to filter out duplicates in Excel.
Count the total number of duplicates in a column(s)The easiest way to count duplicates in a column is to employ any of the formulas we used to identify duplicates in Excel (with or without first occurrences). And then you can count duplicate values by using the following COUNTIF formula:
=COUNTIF(range, "duplicate")
Where “duplicate” is the label you used in the formula that locates duplicates.
In this example, our duplicate formula takes the following shape:
=COUNTIF(B2:B8, "duplicate")
=ROWS($A$2:$A$8)-SUM(IF( COUNTIF($A$2:$A$8,$A$2:$A$8)=1,1,0))
Because it’s an array formula, remember to press Ctrl + Shift + Enter to complete it. Also, please keep in mind that this formula counts all duplicate records, including first occurrences:
To find the total number of duplicate rows, embed the COUNTIFS function instead of COUNTIF in the above formula, and specify all of the columns you want to check for duplicates. For example, to count duplicate rows based on columns A and B, enter the following formula in your Excel sheet:
=ROWS($A$2:$A$8)-SUM(IF( COUNTIFS($A$2:$A$8,$A$2:$A$8, $B$2:$B$8,$B$2:$B$8)=1,1,0))
How to filter duplicates in ExcelFor easier data analysis, you may want to filter your data to only display duplicates. In other situations, you may need the opposite – hide duplicates and view unique records. Below you will find solutions for both scenarios.
How to show and hide duplicates in ExcelTip. To have filtering enabled automatically, convert your data to a fully-functional Excel table. Just select all data and press the
Ctrl + T
shortcut.
And now, you can sort duplicates by the key column to group them for easier analysis. In this example, we can sort duplicate rows by the Order number column:
How to filter duplicates by their occurrencesIf you want to show 2nd, 3rd, or Nth occurrences of duplicate values, use the formula to count duplicate instances we discussed earlier:
=COUNTIF($A$2:$A2, $A2)
Then apply filtering to your table and select only the occurrence(s) you want to view. For example, you can filter the 2nd occurrences like in the following screenshot:
In a similar manner, you can show 2nd, 3rd and all subsequent duplicate occurrences. Just type the required number in the box next to “is greater than”.
Highlight, select, clear, delete, copy or move duplicatesAfter you’ve filtered duplicates like demonstrated above, you have a variety of choices to deal with them.
How to select duplicates in ExcelTo select duplicate records without column headers, select the first (upper-left) cell, and press Ctrl + Shift + End to extend the selection to the last cell.
Tip. In most cases, the above shortcuts work fine and select filtered (visible) rows only. In some rare cases, mostly on very large workbooks, both visible and invisible cells may get selected. To fix this, use one of the above shortcuts first, and then press
Alt + ;
to select only visible cells, ignoring hidden rows.
How to clear or remove duplicates in Excel How to highlight duplicates in ExcelAnother way to highlight duplicates in Excel is using a built-in conditional formatting rule for duplicates, or creating a custom rule specially tailored for your sheet. Experienced Excel users won’t have any problem with creating such a rule based on the formulas we used to check duplicates in Excel. If you are not very comfortable with Excel formulas or rules yet, you will find the detailed steps in this tutorial: How to highlight duplicates in Excel.
How to copy or move duplicates to another sheetTo copy duplicates, select them, press Ctrl + C, then open another sheet (a new or existing one), select the upper-left cell of the range where you want to copy the duplicates, and press Ctrl + V to paste them.
To move duplicates to another sheet, perform the same steps with the only difference that you press Ctrl + X (cut) instead of Ctrl + C (copy).
This is how you can identify duplicates in Excel using functions and built-in features. To better understand the formulas discussed in this tutorial, feel free to download Identify Duplicates Sample Workbook.
Duplicate Remover – fast and efficient way to locate duplicates in ExcelNow that you know how to use duplicate formulas in Excel, let me demonstrate you another quick, efficient and formula-free way – Duplicate Remover for Excel.
This all-in-one tool can search for duplicate or unique values in a single column or compare two columns. It can find, select and highlight duplicate records or entire duplicate rows, remove found dupes, copy or move them to another sheet. I think an example of practical use is worth many words, so let’s get to it.
How to find duplicate rows in Excel in 2 quick stepsTo test the capabilities of our Duplicate Remover add-in, I’ve created a table with a few hundred rows that looks like follows:
As you see, the table has a few columns. The first 3 columns contain the most relevant information, so we are going to search for duplicate rows based solely on the data in columns A – C. To find duplicate records in these columns, just do the following:
The smart add-in will pick up the entire table and ask you to specify the following two things:
Select the columns to check for duplicates (in this example, these are the Order no., Order date and Item columns).
Choose an action to perform on duplicates. Because our purpose is to identify duplicate rows, I’ve selected the Add a status column
Apart from adding a status column, an array of other options are available to you:
Delete duplicates
Color (highlight) duplicates
Select duplicates
Copy duplicates to a new worksheet
Move duplicates to a new worksheet
As you can see in the below screenshot, all of the rows that have identical values in the first 3 columns have been located (first occurrences are not identified as duplicates).
If you want more options to dedupe your worksheets, use the Duplicate Remover wizard that can find duplicates with or without first occurrences as well as unique values. The detailed steps follow below.
Duplicate Remover wizard – more options to search for duplicates in ExcelDepending on a particular sheet you are working with, you may or may not want to treat the first instances of identical records as duplicates. One possible solution is using a different formula for each scenario, as we discussed in How to identify duplicates in Excel. If you are looking for a fast, accurate and formula-free method, try the Duplicate Remover wizard:
On the next step, you are presented with the 4 options to check duplicates in your Excel sheet:
Duplicates without 1st occurrences
Duplicates with 1st occurrences
Unique values
Unique values and 1st duplicate occurrences
For this example, let’s go with the second option, i.e. Duplicates + 1st occurrences:
Now, select the columns where you want to check duplicates. Like in the previous example, we are selecting the first 3 columns:
Finally, choose an action you want to perform on duplicates. As is the case with the Dedupe Table tool, the Duplicate Remover wizard can identify, select, highlight, delete, copy or move duplicates.
It only takes a fraction of a second for the Duplicate Remover wizard to check hundreds of rows, and deliver the following result:
No formulas, no stress, no errors – always swift and impeccable results 🙂
Identify Duplicates – formula examples (.xlsx file) Ultimate Suite – trial version (.zip file)
You may also be interested in
How To Hide And Unhide Rows In Microsoft Excel In 2 Different Ways
10 Things in Tech: Get the latest tech trends & innovations
Loading Something is loading.
Email address
Check out the products mentioned in this article:
Marissa Perino/Business Insider
Highlight adjacent cells to unhide a hidden row.
Marissa Perino/Business Insider
Unhide all rows.
Marissa Perino/Business Insider
Related coverage from How To Do Everything: Tech:
Cập nhật thông tin chi tiết về 7 Ways To Find And Remove Duplicate Values In Microsoft Excel trên website Hoisinhvienqnam.edu.vn. Hy vọng nội dung bài viết sẽ đáp ứng được nhu cầu của bạn, chúng tôi sẽ thường xuyên cập nhật mới nội dung để bạn nhận được thông tin nhanh chóng và chính xác nhất. Chúc bạn một ngày tốt lành!