Uses of Class
com.bytedesk.kbase.llm_website.crawl.WebsiteCrawlConfig

Packages that use WebsiteCrawlConfig

Package	Description
com.bytedesk.kbase.llm_website
com.bytedesk.kbase.llm_website.crawl
com.bytedesk.kbase.llm_website.service

Uses of WebsiteCrawlConfig in com.bytedesk.kbase.llm_website

Fields in com.bytedesk.kbase.llm_website declared as WebsiteCrawlConfig

Modifier and Type	Field	Description
private @NotNull(message="\u6293\u53d6\u914d\u7f6e\u4e0d\u80fd\u4e3a\u7a7a") @Valid WebsiteCrawlConfig	WebsiteCrawlConfigRequest.config
private WebsiteCrawlConfig	WebsiteCrawlRequest.config

Methods in com.bytedesk.kbase.llm_website that return WebsiteCrawlConfig

Modifier and Type	Method	Description
WebsiteCrawlConfig	WebsiteRestService.getCrawlConfig(String websiteUid)	获取网站抓取配置

Methods in com.bytedesk.kbase.llm_website with parameters of type WebsiteCrawlConfig

Modifier and Type	Method	Description
CompletableFuture<WebsiteCrawlResult>	WebsiteRestService.startCrawl(String websiteUid, WebsiteCrawlConfig config)	开始整站抓取
WebsiteResponse	WebsiteRestService.updateCrawlConfig(String websiteUid, WebsiteCrawlConfig config)	更新网站抓取配置

Uses of WebsiteCrawlConfig in com.bytedesk.kbase.llm_website.crawl

Methods in com.bytedesk.kbase.llm_website.crawl that return WebsiteCrawlConfig

Modifier and Type	Method	Description
WebsiteCrawlConfig	WebsiteCrawlTask.getConfig()	获取抓取配置
static WebsiteCrawlConfig	WebsiteCrawlConfig.getDeep()	获取深度配置（更大深度和页面数）
static WebsiteCrawlConfig	WebsiteCrawlConfig.getDefault()	获取默认配置
static WebsiteCrawlConfig	WebsiteCrawlConfig.getFast()	获取快速配置（较少深度和页面数）

Methods in com.bytedesk.kbase.llm_website.crawl with parameters of type WebsiteCrawlConfig

Modifier and Type	Method	Description
void	WebsiteCrawlTask.setConfig(WebsiteCrawlConfig config)	设置抓取配置

Uses of WebsiteCrawlConfig in com.bytedesk.kbase.llm_website.service

Methods in com.bytedesk.kbase.llm_website.service with parameters of type WebsiteCrawlConfig

Modifier and Type	Method	Description
private boolean	WebsiteCrawlerService.crawlSinglePage(String url, WebsiteEntity website, Set<String> visitedUrls, Set<String> urlsToVisit, WebsiteCrawlConfig config)	抓取单个页面
private WebsiteCrawlTask	WebsiteCrawlerService.createCrawlTask(WebsiteEntity website, WebsiteCrawlConfig config)	创建抓取任务
private void	WebsiteCrawlerService.extractLinks(org.jsoup.nodes.Document doc, String currentUrl, String baseUrl, Set<String> urlsToVisit, WebsiteCrawlConfig config)	提取页面链接
private boolean	WebsiteCrawlerService.isValidContent(String content, WebsiteCrawlConfig config)	验证内容是否有效
private WebsiteCrawlResult	WebsiteCrawlerService.performCrawl(WebsiteEntity website, WebsiteCrawlTask task, WebsiteCrawlConfig config)	执行抓取任务
private boolean	WebsiteCrawlerService.shouldCrawlUrl(String url, URL baseUrl, WebsiteCrawlConfig config)	判断URL是否应该被抓取
CompletableFuture<WebsiteCrawlResult>	WebsiteCrawlerService.startCrawl(String websiteUid, WebsiteCrawlConfig config)	开始整站抓取