public class BuildHostMap extends Object
run(BufferedReader, PrintStream, DataOutputStream, DataOutputStream, boolean, ProgressLogger).
Warning: this class provides a main method that saves the host list to standard output, but it does some logging, too, so be careful not to log to standard output.
|Constructor and Description|
|Modifier and Type||Method and Description|
This method reads URLs and writes hosts (or, possibly, top private domains), together with a map from URLs to hosts and a host count.
public static final Pattern DOTTED_ADDRESS
public static void run(BufferedReader br, PrintStream hosts, DataOutputStream mapDos, DataOutputStream countDos, boolean topPrivateDomain, ProgressLogger pl) throws IOException, URISyntaxException
br- the buffered reader returning the list of URLs.
hosts- the print stream where hosts will be printed.
mapDos- the data output stream where the map from URLs to hosts will be written (one integer per URL).
countDos- the data output stream where the host counts will be written (one integer per host).
topPrivateDomain- if true, we use
InternetDomainName.topPrivateDomain()to map to top private domains, rather than hosts.
pl- a progress logger, or