我有两个 3GB 的文本文件，每个文件大约有 8000 万行.并且它们共享 99.9% 的相同行(文件 A 有 60,000 个唯一行，文件 B 有 80,000 个唯一行).
I have two 3GB text files, each file has around 80 million lines. And they share 99.9% identical lines (file A has 60,000 unique lines, file B has 80,000 unique lines).
How can I quickly find those unique lines in two files? Is there any ready-to-use command line tools for this? I'm using Python but I guess it's less possible to find a efficient Pythonic method to load the files and compare.
欢迎提出任何建议.
推荐答案
如果顺序很重要，请尝试 comm 实用程序.如果顺序无关紧要，sort file1 file2 |uniq -u.
If order matters, try the comm utility. If order doesn't matter, sort file1 file2 | uniq -u.
