Randomize the line numbers in a text file:
python -c "import random, sys; lines = open(sys.argv[1]).readlines(); random.shuffle(lines); print ''.join(lines)," D:\GPC\plfiles\Desktop\KPT\Corpus\OtherLangCorpus\Sindhi\combined\test.txt > D:\GPC\plfiles\Desktop\KPT\Corpus\OtherLangCorpus\Sindhi\combined\test_rand.txt


________
Count Unique lines in a large file (works on Ubuntu):
sort input.txt | uniq -c > output.txt

Sorts file with highest frequency lines at the top:
sort input.txt | uniq -c | sort -bgr > output.txt
_______________
Compare text Files:
Find line numbers in second file that are not present in the first file.
i.e. compare first and second file and give list of the words from second file that is not present in the first file.
Basically looks for all lines in second-file.txt which don't match any line in first-file.txt. Might be slow if the files are large.



grep -Fxv -f first-file.txt second-file.txt

_________

Notepad++

Referential RegxReplace

Add a tab after a find string (e.g. after the first group of numbers at each line)

Find: (^\d+)

Replace: \1\t

___

Alll Devanagari (Hindi/Konkani) characters that can be found via a regular expressions :

ँ|ं|ः|अ |आ|इ|ई|उ|ऊ|ए|ऐ|ओ|औ|क्ष|क|ख|ग|घ|च|छ|ज्ञ|ज|झ|ञ|ट|ठ|ड|ढ|ण|त|थ|द|ध|न|प|फ|ब|भ|म|य|र|ल|ळ|व|श|ष|स|ह|़|ा|ि|ी|ु|ू|ृ|े|ै|ॉ|ो|ौ|्|ॐ

____

 Find Invalid Words that have two consquent invalid diacritics in Hindi (for Notepad++):

[ा|ि|ी|ु|ू|ृ|े|ै|ॉ|ो|ौ|्][ा|ि|ी|ु|ू|ृ|े|ै|ॉ|ो|ौ|्]

 

Find non-ASCII values (which is roughly unicode values:

[^\x00-\x7F]+

 

 

 

Add comment


Security code
Refresh