Java Code Examples for org.openqa.selenium.htmlunit.HtmlUnitDriver#get()

The following examples show how to use org.openqa.selenium.htmlunit.HtmlUnitDriver#get() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: DemoSeleniumCrawler.java    From WebCollector with GNU General Public License v3.0 6 votes vote down vote up 
public static void main(String[] args) throws Exception {
    Executor executor = new Executor() {
        @Override
        public void execute(CrawlDatum datum, CrawlDatums next) throws Exception {
            
            HtmlUnitDriver driver = new HtmlUnitDriver();
            driver.setJavascriptEnabled(true);
            
            driver.get(datum.url());
            
            List<WebElement> elementList = driver.findElementsByCssSelector("h3.vrTitle a");
            for(WebElement element:elementList){
                System.out.println("title:"+element.getText());
            }
        }
    };

    //创建一个基于伯克利DB的DBManager
    DBManager manager = new RocksDBManager("crawl");
    //创建一个Crawler需要有DBManager和Executor
    Crawler crawler = new Crawler(manager, executor);
    crawler.addSeed("https://www.sogou.com/web?query=%E6%B7%98%E5%AE%9D");
    crawler.start(1);
}
Example 2
Source File: Snexus.java    From BoxHelper with GNU General Public License v3.0 5 votes vote down vote up 
public Snexus(String page,  HashMap<String, String> cookies, String passkey, float downLimit, float upLimit) {
        this.page = page;
        this.cookies = cookies;
        this.passkey = passkey;
        this.upLimit = upLimit;
        this.downLimit = downLimit;
//        String pageToGetPasskey = page;
        this.domain = page.substring(page.indexOf("//") + 2, page.indexOf("/", page.indexOf("//") + 2));
        HtmlUnitDriver driver = new HtmlUnitDriver();
        driver.get("https://" + getDomain());
        getCookies().forEach((k, v) -> driver.manage().addCookie(new Cookie(k, v)));
        setDriver(driver);
    }
Example 3
Source File: Snexus.java    From BoxHelper with GNU General Public License v3.0 5 votes vote down vote up 
public HashSet<RawTorrent> acquireTorrents(){
    HtmlUnitDriver driver = this.getDriver();
    HashSet<RawTorrent> torrents = new HashSet<>();
    driver.get(getPage());
    String pageSource = driver.getPageSource();
    torrents = decodePageSource(pageSource.substring(pageSource.lastIndexOf("colhead")));
    return torrents;
}
Example 4
Source File: Snexus.java    From BoxHelper with GNU General Public License v3.0 5 votes vote down vote up 
public String acquireHash(int id){
    HtmlUnitDriver driver = this.getDriver();
    driver.get("https://" + this.domain + "/details.php?id=" + id);
    String pageSource = driver.getPageSource();
    String reg = "[a-zA-Z0-9]{32}";
    Pattern pattern = Pattern.compile(reg);
    if (pageSource.contains("Hash")) {
        Matcher matcher = pattern.matcher(pageSource.substring(pageSource.indexOf("Hash")));
        if (matcher.find()) return matcher.group();
        else return "";
    } else return "";
}
Example 5
Source File: HiddenBrowserDriver.java    From ats-framework with Apache License 2.0 5 votes vote down vote up 
@Override
@PublicAtsApi
public void start() {

    webDriver = new HtmlUnitDriver(this.browserVersion);
    webDriver.setJavascriptEnabled(true);

    setProxyIfAvailable();

    fixHtmlUnitBehaviour();

    log.info("Opening URL: " + url);
    webDriver.get(url);
}
Example 6
Source File: LocalWebDriverTest.java    From Mastering-Software-Testing-with-JUnit-5 with MIT License 5 votes vote down vote up 
@Test
public void testWithHeadlessBrowsers(HtmlUnitDriver htmlUnit,
        PhantomJSDriver phantomjs) {
    htmlUnit.get("https://bonigarcia.github.io/selenium-jupiter/");
    phantomjs.get("https://bonigarcia.github.io/selenium-jupiter/");

    assertTrue(htmlUnit.getTitle().contains("JUnit 5 extension"));
    assertNotNull(phantomjs.getPageSource());
}
Example 7
Source File: HtmlUnitJupiterTest.java    From selenium-jupiter with Apache License 2.0 4 votes vote down vote up 
@Test
public void test(HtmlUnitDriver driver) {
    driver.get("https://bonigarcia.github.io/selenium-jupiter/");
    assertThat(driver.getTitle(),
            containsString("JUnit 5 extension for Selenium"));
}