Saturday, March 28, 2015

Sort a set of strings by each string's last character, solution 2

Here's another solution to the same problem, this time I am not using a comparator. Instead, I reverse each element, save it in a new TreeSet which keeps elements in sorted order, then instantiate a new LinkedHashSet which maintains an insertion order, reverse the strings again to the way there were. What I noticed doing this problem is that this solution is faster with larger sample size than solution with comparator. That's a surprising revelation. Also, using StringBuilder to reverse a string is a lot faster than reversing character by character.
 package com.ervits;  
 import java.util.Arrays;  
 import java.util.HashSet;  
 import java.util.LinkedHashSet;  
 import java.util.Objects;  
 import java.util.Set;  
 import java.util.TreeSet;  
 /**  
  *  
  * @author Artem  
  * @created Mar 27, 2015 2:15:49 PM  
  *  
  * Project SetSorter  
  *  
  */  
 public class SetSorterWithReverseString {  
   public static void main(String[] args) {  
     long start = System.currentTimeMillis();  
     Set<String> unsorted = new HashSet<>();  
     unsorted.addAll(Arrays.asList("Abstract", "class", "is", "a", "special",   
         "class", "in", "Java", "it", "can", "not", "be", "instantiated",   
         "and", "that's", "why", "can", "not", "be", "used", "directly",  
         "At", "first", "concept", "of", "abstraction", "abstract",   
         "class", "and", "interface", "all", "look", "useless", "to",   
         "many", "developers", "because", "you", "can", "not", "implement",   
         "any", "method", "in", "interface", "you", "can", "not", "create", "object",  
         "To", "create", "these", "etexts", "the", "Project", "expends", "considerable",  
         "efforts", "to", "identify", "transcribe", "and", "proofread", "public", "domain",  
         "Take", "the", "hue", "of", "the", "company", "you", "are", "with", "Take",   
         "characters", "as", "they", "do", "most", "things", "upon", "trust",   
         "Take", "rather", "than", "give", "the", "tone", "of", "the", "company",   
         "you", "are", "in", "Take", "nothing", "for", "granted", "upon", "the",   
         "bare", "authority", "of", "the", "author", "Taking", "up", "adventitious",   
         "proves", "their", "want", "of", "intrinsic", "merit", "Talent", "of", "hating",   
         "with", "good_breeding", "and", "loving", "with", "prudence", "Talk", "often",   
         "but", "never", "long", "Talk", "sillily", "upon", "a", "subject", "of", "other",   
         "people's", "Talk", "of", "natural", "affection", "is", "talking", "nonsense",   
         "Talking", "of", "either", "your", "own", "or", "other", "people's", "domestic",   
         "affairs", "Tell", "me", "whom", "you", "live", "with", "and", "I", "will",   
         "tell", "you", "who", "you", "are", "Tell", "stories", "very", "seldom", "The",   
         "longest", "life", "is", "too", "short", "for", "knowledge", "The", "present",   
         "moments", "are", "the", "only", "ones", "we", "are", "sure", "of", "The", "best",   
         "have", "something", "bad", "and", "something", "little", "The", "worst", "have",   
         "something", "good", "and", "sometimes", "something", "great", "There", "are",   
         "many", "avenues", "to", "every", "man", "They", "thought", "I", "informed",   
         "because", "I", "pleased", "them", "Thin", "veil", "of", "Modesty", "drawn",   
         "before", "Vanity", "Think", "to", "atone", "by", "zeal", "for", "their",   
         "want", "of", "merit", "and", "importance", "Think", "yourself", "less",   
         "well", "than", "you", "are", "in", "order", "to", "be", "quite", "so",   
         "Thinks", "himself", "much", "worse", "than", "he", "is", "Thoroughly",   
         "not", "superficially", "Those", "who", "remarkably", "affect", "any",   
         "one", "virtue", "Those", "whom", "you", "can", "make", "like", "themselves",   
         "better", "Three", "passions", "that", "often", "put", "honesty", "to",   
         "most", "severe", "trials", "Timidity", "and", "diffidence", "To", "be",   
         "heard", "with", "success", "you", "must", "be", "heard", "with", "pleasure",   
         "To", "be", "pleased", "one", "must", "please", "To", "govern", "mankind",   
         "one", "must", "not", "overrate", "them", "To", "seem", "to", "have", "forgotten",   
         "what", "one", "remembers", "To", "know", "people's", "real", "sentiments",   
         "I", "trust", "much", "more", "to", "my", "eyes", "To", "great", "caution",   
         "you", "can", "join", "seeming", "frankness", "and", "openness", "Too", "like",   
         "and", "too", "exact", "a", "picture", "of", "human", "nature", "Trifle", "only",   
         "with", "triflers", "and", "be", "serious", "only", "with", "the", "serious",   
         "Trifles", "that", "concern", "you", "are", "not", "trifles", "to", "me",   
         "Trifling", "parts", "with", "their", "little", "jargon", "Trite", "jokes",   
         "and", "loud", "laughter", "reduce", "him", "to", "a", "buffoon", "Truth",   
         "but", "not", "the", "whole", "truth", "must", "be", "the", "invariable",   
         "principle", "Truth", "leaves", "no", "room", "for", "compliments", "Unaffected",   
         "silence", "upon", "that", "subject", "is", "the", "only", "true", "medium",   
         "Unguarded", "frankness", "Unintelligible", "to", "his", "readers", "and",   
         "sometimes", "to", "himself", "Unopened", "because", "one", "title", "in",   
         "twenty", "has", "been", "omitted", "Unwilling", "and", "forced", "it",   
         "will", "never", "please", "Use", "palliatives", "when", "you", "contradict",   
         "Useful", "sometimes", "to", "see", "the", "things", "which", "one", "ought",   
         "to", "avoid", "Value", "of", "moments", "when", "cast", "up", "is", "immense",   
         "Vanity", "interest", "and", "absurdity", "always", "display", "Vanity", "that",   
         "source", "of", "many", "of", "our", "follies", "Warm", "and", "young", "thanks",   
         "not", "old", "and", "cold", "ones", "Water_drinkers", "can", "write", "nothing",   
         "good", "We", "love", "to", "be", "pleased", "better", "than", "to", "be",   
         "informed", "We", "have", "many", "of", "those", "useful", "prejudices", "in",   
         "this", "country", "We", "shall", "be", "feared", "if", "we", "do", "not",   
         "show", "that", "we", "fear", "Well", "dressed", "not", "finely", "dressed",   
         "What", "pleases", "you", "in", "others", "will", "in", "general", "please",   
         "them", "in", "you", "What", "displeases", "or", "pleases", "you", "in",   
         "others", "What", "you", "feel", "pleases", "you", "in", "them", "What",   
         "have", "I", "done", "to_day", "What", "is", "impossible", "and", "what",   
         "is", "only", "difficult", "Whatever", "pleases", "you", "most", "in",   
         "others", "Whatever", "is", "worth", "doing", "at", "all", "is", "worth",   
         "doing", "well", "Whatever", "one", "must", "do", "one", "should", "do",   
         "de", "bonne", "grace", "Whatever", "real", "merit", "you", "have",   
         "other", "people", "will", "discover", "When", "well", "dressed",   
         "for", "the", "day", "think", "no", "more", "of", "it", "afterward",   
         "Where", "one", "would", "gain", "people", "remember", "that", "nothing",   
         "is", "little", "Who", "takes", "warning", "by", "the", "fate", "of",   
         "others", "Wife", "very", "often", "heard", "indeed", "but", "seldom",   
         "minded", "Will", "not", "so", "much", "as", "hint", "at", "our", "follies",   
         "Will", "pay", "very", "dear", "for", "the", "quarrels", "and", "ambition",   
         "of", "a", "few", "Wish", "you", "my", "dear", "friend", "as", "many",   
         "happy", "new", "years", "as", "you", "deserve", "Wit", "may", "created",   
         "any", "admirers", "but", "makes", "few", "friends", "Witty", "without",   
         "satire", "or", "commonplace", "Woman", "like", "her", "who", "has",   
         "always", "pleased", "and", "often", "been", "pleased", "Women", "are",   
         "the", "only", "refiners", "of", "the", "merit", "of", "men", "Women",   
         "choose", "their", "favorites", "more", "by", "the", "ear", "Women",   
         "are", "all", "so", "far", "Machiavelians", "Words", "are", "the",   
         "dress", "of", "thoughts", "World", "is", "taken", "by", "the",   
         "outside", "of", "things", "Would", "not", "tell", "what", "she",   
         "did", "not", "know", "Wrapped", "up", "and", "absorbed", "in",   
         "their", "abstruse", "speculations", "Writing", "anything", "that",   
         "may", "deserve", "to", "be", "read", "Writing", "what", "may",   
         "deserve", "to", "be", "read", "Wrongs", "are", "often",   
         "forgiven", "but", "contempt", "never", "is", "Yielded", "commonly",   
         "without", "conviction", "You", "must", "be", "respectable", "if",   
         "you", "will", "be", "respected", "You", "had", "much", "better",   
         "hold", "your", "tongue", "than", "them", "Young", "people", "are",   
         "very", "apt", "to", "overrate", "both", "men", "and", "things",   
         "Young", "fellow", "ought", "to", "be", "wiser", "than", "he",   
         "should", "seem", "to", "be", "Young", "men", "are", "as", "apt",   
         "to", "think", "themselves", "wise", "enough", "Your", "merit",   
         "and", "your", "manners", "can", "alone", "raise", "you", "Your",   
         "character", "there", "whatever", "it", "is", "will", "get",   
         "before", "you", "here"));  
     Set<String> sorted = reverseAndSort(unsorted);  
     System.out.println("***** UNSORTED *****");  
     System.out.println(unsorted.toString());  
     System.out.println("***** SORTED *****");  
     System.out.println(sorted.toString());  
     long end = System.currentTimeMillis();  
     System.out.printf("Time taken: %d\n", (end - start));  
     System.out.printf("Unsorted size: %d\n", unsorted.size());  
     System.out.printf("Sorted size: %d\n", sorted.size());  
   }  
   public static Set<String> reverseAndSort(Set<String> unsorted) {  
     Set<String> sorted = new TreeSet<>();  
     for (String aStr : unsorted) {  
       if(aStr != null) {  
         sorted.add(reverseWithStringBuilder(aStr));  
       }    
     }  
     return normalizeSet(sorted);  
   }  
   static String reverseString(String aStr) {  
     String reversed = "";  
     for (int i = aStr.length() - 1; i >= 0; i--) {  
       reversed += aStr.charAt(i);  
     }  
     return reversed;  
   }  
   static Set<String> normalizeSet(Set<String> set) {  
     Set<String> finalSet = new LinkedHashSet<>();  
     Objects.requireNonNull(set, "passed set is null");  
     for (String aStr : set) {  
       if(aStr != null) {  
         finalSet.add(reverseWithStringBuilder(aStr));  
       }  
     }  
     return finalSet;  
   }  
   // Faster than reverseString()  
   static String reverseWithStringBuilder(String aStr) {  
     Objects.requireNonNull(aStr, "passed String cannot be null");  
     return new StringBuilder(aStr).reverse().toString();  
   }  
 }  

Wednesday, March 25, 2015

Sort a set of strings by each string's last character, solution using comparator (Update)

I have the following problem. Sort a set of strings by each string's last character. This is my implementation, please suggest other solutions.

UPDATE
turns out I had a bug in my previous version, I was only comparing the last character to last character and on the return from the method, it would ignore every other word with same last character. It was only noticeable after a much larger sample set.



 package com.ervits;  
 import java.util.Comparator;  
 import java.util.Objects;  
 /**  
  *  
  * @author Artem  
  * @created Mar 27, 2015 2:07:21 PM  
  *  
  * Project SetSorter  
  *  
  */  
 class LastCharComparator implements Comparator {  
   public LastCharComparator() {  
   }  
   @Override  
   public int compare(String t1, String t2) {  
     Objects.requireNonNull(t1, "passed String1 cannot be null");  
     Objects.requireNonNull(t2, "passed String2 cannot be null");  
     final int result;  
     if(reverseWithStringBuilder(t1)  
         .compareTo(reverseWithStringBuilder(t2)) &gt; 0) {  
       result = 1;  
     } else if (reverseWithStringBuilder(t1)  
         .compareTo(reverseWithStringBuilder(t2)) &lt; 0) {  
       result = -1;  
     } else {  
       result = 0;  
     }  
     return result;  
   }  
   static String reverseWithStringBuilder(String aStr) {  
     return new StringBuilder(aStr).reverse().toString();  
   }  
 }  
 package com.ervits;  
 import java.util.Arrays;  
 import java.util.Comparator;  
 import java.util.LinkedHashSet;  
 import java.util.Set;  
 import java.util.SortedSet;  
 import java.util.TreeSet;  
 /**  
  *  
  * @author artem  
  */  
 public class SetSorter {  
   public static void main(String[] args) {  
     Set unsorted = new LinkedHashSet&lt;&gt;();  
     unsorted.addAll(Arrays.asList("xxxxxX", "xxxxxY", "hell7oD", "hel1loB", "helloC", "helloA", "helloE", "helloA", "helloZ"));  
     Comparator setComparator = new LastCharComparator();  
     SortedSet sorted = new TreeSet&lt;&gt;(setComparator);  
     sorted.addAll(unsorted);  
     System.out.println("***** UNSORTED *****");  
     System.out.println(unsorted.toString());  
     System.out.println("***** SORTED *****");  
     System.out.println(sorted.toString());  
     System.out.println("***** COMPARATOR *****");  
     System.out.println(sorted.comparator().getClass().getCanonicalName());  
   }  
 }  

Thursday, March 5, 2015

Running saved sqoop 1.4.4 jobs with password file

This has been bugging me for a long time. I've been trying to pass password to sqoop job securely for the longest time with no avail. Apparently, my approach was flawed by two things. First of all, the sqoop user guide 1.4.4 has incorrect spelling for password file option in the example. The documentation has the correct option but if you're like me and follow the example you will get it wrong.

wrong way in Sqoop 1.4.4 user guide:
sqoop import --connect jdbc:mysql://database.example.com/employees \
    --username venkatesh --passwordFile ${user.home}/.password

right way in Sqoop 1.4.5 user guide:
sqoop import --connect jdbc:mysql://database.example.com/employees \
    --username venkatesh --password-file ${user.home}/.password

the other flaw is that when you create the password file with "echo" command, make sure you don't append a new line character to the password. Here's a handy script to create a password:

echo -n "password" > .password
hdfs dfs -put .password /user/$USER/
hdfs dfs -chmod 400 /user/$USER/.password
rm .password

And now when you create a job, you can pass --password-file option and the /user/$USER/.password as the location and it will work.

The fix for flaw #2 I found in this Stack Overflow post.