Java Code Examples for us.codecraft.webmagic.Request#setMethod()

The following examples show how to use us.codecraft.webmagic.Request#setMethod() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: CovertUtil.java    From vscrawler with Apache License 2.0 5 votes vote down vote up 
public static Request convertSeed(Seed seed) {
    Request request = new Request(seed.getData());
    request.setMethod("GET");
    Map<String, Object> ext = Maps.newHashMap();
    ext.putAll(seed.getExt());
    request.setExtras(ext);
    return request;
}
Example 2
Source File: DuplicateRemovedSchedulerTest.java    From webmagic with Apache License 2.0 5 votes vote down vote up 
@Test
public void test_no_duplicate_removed_for_post_request() throws Exception {
    DuplicateRemover duplicateRemover = Mockito.mock(DuplicateRemover.class);
    duplicateRemovedScheduler.setDuplicateRemover(duplicateRemover);
    Request request = new Request("https://www.google.com/");
    request.setMethod(HttpConstant.Method.POST);
    duplicateRemovedScheduler.push(request, null);
    verify(duplicateRemover,times(0)).isDuplicate(any(Request.class),any(Task.class));
}
Example 3
Source File: DuplicateRemovedSchedulerTest.java    From webmagic with Apache License 2.0 5 votes vote down vote up 
@Test
public void test_duplicate_removed_for_get_request() throws Exception {
    DuplicateRemover duplicateRemover = Mockito.mock(DuplicateRemover.class);
    duplicateRemovedScheduler.setDuplicateRemover(duplicateRemover);
    Request request = new Request("https://www.google.com/");
    request.setMethod(HttpConstant.Method.GET);
    duplicateRemovedScheduler.push(request, null);
    verify(duplicateRemover,times(1)).isDuplicate(any(Request.class),any(Task.class));
}