2026-08-18 00:50:44 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: sitebunny)
2026-08-18 00:50:44 [scrapy.utils.log] INFO: Versions: lxml 4.9.3.0, libxml2 2.10.3, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.2, Twisted 22.10.0, Python 3.11.13 (main, Jul 22 2025, 01:43:09) [GCC 10.2.1 20210110], pyOpenSSL 23.2.0 (OpenSSL 3.1.2 1 Aug 2023), cryptography 41.0.3, Platform Linux-5.15.0-141-generic-x86_64-with-glibc2.31
2026-08-18 00:50:44 [scrapy.addons] INFO: Enabled addons:
[]
2026-08-18 00:50:44 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'sitebunny',
 'CLOSESPIDER_PAGECOUNT': '2001',
 'CONCURRENT_REQUESTS': '4',
 'COOKIES_ENABLED': False,
 'DEPTH_LIMIT': '10',
 'DEPTH_PRIORITY': 1,
 'DOWNLOAD_DELAY': 1,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/data/webcrawl/scrapyd.dev/logs/sitebunny/sb-spider-v1/d4c3d24a9a9e11f1ba910242ac130002.log',
 'LOG_LEVEL': 'INFO',
 'NEWSPIDER_MODULE': 'sitebunny.spiders',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'ROBOTSTXT_OBEY': 'False',
 'SCHEDULER_DISK_QUEUE': 'scrapy.squeues.PickleFifoDiskQueue',
 'SCHEDULER_MEMORY_QUEUE': 'scrapy.squeues.FifoMemoryQueue',
 'SPIDER_MODULES': ['sitebunny.spiders'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'}
2026-08-18 00:50:44 [scrapy.extensions.telnet] INFO: Telnet Password: 056b58e53b8eb431
2026-08-18 00:50:44 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.closespider.CloseSpider',
 'scrapy.extensions.logstats.LogStats']
2026-08-18 00:50:44 [sb-spider-v1] INFO: {'allow': 'https?://www.42maru.ai/kr', 'deny': 'signin|sign-in|Signin|SignIn|signIn|login|log-in|Login|LogIn|logIn|signup|sign-up|Signup|SignUp|signUp|findid|find-id|FindId|FindID|findId|findID|findpw|find-pw|FindPw|FindPW|findPw|findPW|findpassword|find-password|FindPassword|findPassword|findaccount|find-account|FindAccount|findAccount|/idpw|forgot-password|change-password|mode=download|downloadFile|/file/download|download.php|/download|/bbs/down.php|attachFile/download|getVcard.|dispBoardTagList|.(jpg|pdf|hwp|xls|xlsx|doc|docx|zip)$'}
2026-08-18 00:50:44 [sb-spider-v1] INFO: {'link_extractor': <scrapy.linkextractors.lxmlhtml.LxmlLinkExtractor object at 0x7faec44c7e90>, 'callback': 'parse_item', 'follow': True, 'process_request': 'set_playwright', 'errback': 'errback'}
2026-08-18 00:50:44 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-08-18 00:50:44 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-08-18 00:50:44 [scrapy.middleware] INFO: Enabled item pipelines:
['sitebunny.pipelines.SitebunnyPipeline']
2026-08-18 00:50:44 [scrapy.core.engine] INFO: Spider opened
2026-08-18 00:50:44 [scrapy.extensions.closespider] INFO: Spider will stop when no items are produced after 300 seconds.
2026-08-18 00:50:44 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-08-18 00:50:44 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6024
2026-08-18 00:50:44 [scrapy-playwright] INFO: Starting download handler
2026-08-18 00:50:44 [scrapy-playwright] INFO: Starting download handler
2026-08-18 00:50:49 [sb-spider-v1] INFO: Message sent successfully to Slack!
2026-08-18 00:50:49 [scrapy-playwright] INFO: Launching browser chromium
2026-08-18 00:50:50 [scrapy-playwright] INFO: Browser chromium launched
2026-08-18 00:50:58 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/ None None
2026-08-18 00:51:11 [root] INFO: 신규 페이지 저장
2026-08-18 00:51:18 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/ 1 1
2026-08-18 00:51:24 [root] INFO: 기수집. 미업데이트. update last_crawled_at
2026-08-18 00:51:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/ 2 2
2026-08-18 00:51:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/Notice/ 2 2
2026-08-18 00:51:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/ 2 2
2026-08-18 00:51:37 [root] INFO: 신규 페이지 저장
2026-08-18 00:51:49 [root] INFO: 신규 페이지 저장
2026-08-18 00:51:49 [scrapy.extensions.logstats] INFO: Crawled 5 pages (at 5 pages/min), scraped 4 items (at 4 items/min)
2026-08-18 00:51:55 [root] INFO: 기수집. 미업데이트. update last_crawled_at
2026-08-18 00:51:59 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/contact/ 5 5
2026-08-18 00:52:08 [root] INFO: 신규 페이지 저장
2026-08-18 00:52:09 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/%EB%8B%B9%EC%82%AC-%EC%82%AC%EC%B9%AD-%EC%A3%BC%EC%9D%98-%EC%95%88%EB%82%B4/ 6 6
2026-08-18 00:52:09 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/%EC%9E%85%EC%B0%B0-%EA%B3%B5%EA%B3%A0-ai-%EB%AA%A8%EB%8D%B8-%ED%95%99%EC%8A%B5%EC%9D%84-%EC%9C%84%ED%95%9C-gpu%EC%84%9C%EB%B2%84-%EA%B5%AC%EB%A7%A4/ 6 6
2026-08-18 00:52:20 [root] INFO: 신규 페이지 저장
2026-08-18 00:52:32 [root] INFO: 신규 페이지 저장
2026-08-18 00:52:36 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/ 8 8
2026-08-18 00:52:40 [root] INFO: 기수집. 미업데이트. update last_crawled_at
2026-08-18 00:52:41 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42MaruWIS/ 9 9
2026-08-18 00:52:41 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=ANSWERINGAI 9 9
2026-08-18 00:52:52 [root] INFO: 신규 페이지 저장
2026-08-18 00:52:52 [scrapy.extensions.logstats] INFO: Crawled 11 pages (at 6 pages/min), scraped 10 items (at 6 items/min)
2026-08-18 00:53:04 [root] INFO: 신규 페이지 저장
2026-08-18 00:53:04 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=CHAT42 11 11
2026-08-18 00:53:15 [root] INFO: 신규 페이지 저장
2026-08-18 00:53:20 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=WorldITShow 12 12
2026-08-18 00:53:32 [root] INFO: 신규 페이지 저장
2026-08-18 00:53:33 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=WIS2022 13 13
2026-08-18 00:53:33 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=WIS 13 13
2026-08-18 00:53:33 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%9B%94%EB%93%9CIT%EC%87%BC 13 13
2026-08-18 00:53:44 [root] INFO: 신규 페이지 저장
2026-08-18 00:53:55 [root] INFO: 신규 페이지 저장
2026-08-18 00:53:55 [scrapy.extensions.logstats] INFO: Crawled 16 pages (at 5 pages/min), scraped 15 items (at 5 items/min)
2026-08-18 00:54:07 [root] INFO: 신규 페이지 저장
2026-08-18 00:54:13 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AI%ED%94%8C%EB%9E%AB%ED%8F%BC 16 16
2026-08-18 00:54:25 [root] INFO: 신규 페이지 저장
2026-08-18 00:54:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AI 17 17
2026-08-18 00:54:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%ED%8F%AC%ED%8B%B0%ED%88%AC%EB%A7%88%EB%A3%A8 17 17
2026-08-18 00:54:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42MaruAIEXPO/ 17 17
2026-08-18 00:54:36 [root] INFO: 신규 페이지 저장
2026-08-18 00:54:50 [root] INFO: 신규 페이지 저장
2026-08-18 00:54:50 [scrapy.extensions.logstats] INFO: Crawled 20 pages (at 4 pages/min), scraped 19 items (at 4 items/min)
2026-08-18 00:55:01 [root] INFO: 신규 페이지 저장
2026-08-18 00:55:08 [sb-spider-v1] INFO: request callback: 500, request_sub_id: 0, processed_item_count: 20
2026-08-18 00:55:08 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EA%B5%AD%EC%A0%9C%EC%9D%B8%EA%B3%B5%EC%A7%80%EB%8A%A5%EB%8C%80%EC%A0%84 20 20
2026-08-18 00:55:20 [root] INFO: 신규 페이지 저장
2026-08-18 00:55:20 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AIEXPO2022 21 21
2026-08-18 00:55:20 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AIEXPO 21 21
2026-08-18 00:55:20 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AIEXPOKOREA 21 21
2026-08-18 00:55:32 [root] INFO: 신규 페이지 저장
2026-08-18 00:55:43 [root] INFO: 신규 페이지 저장
2026-08-18 00:55:55 [root] INFO: 신규 페이지 저장
2026-08-18 00:55:55 [scrapy.extensions.logstats] INFO: Crawled 24 pages (at 4 pages/min), scraped 24 items (at 5 items/min)
2026-08-18 00:56:02 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/CHAT42GS/ 24 24
2026-08-18 00:56:13 [root] INFO: 신규 페이지 저장
2026-08-18 00:56:13 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=GS%EC%9D%B8%EC%A6%9D 25 25
2026-08-18 00:56:25 [root] INFO: 신규 페이지 저장
2026-08-18 00:56:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=1%EB%93%B1%EA%B8%89 26 26
2026-08-18 00:56:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AI%EA%B8%B0%EB%B0%98%EC%86%94%EB%A3%A8%EC%85%98 26 26
2026-08-18 00:56:37 [root] INFO: 신규 페이지 저장
2026-08-18 00:56:48 [root] INFO: 신규 페이지 저장
2026-08-18 00:56:48 [scrapy.extensions.logstats] INFO: Crawled 28 pages (at 4 pages/min), scraped 28 items (at 4 items/min)
2026-08-18 00:56:54 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%B1%9742 28 28
2026-08-18 00:57:05 [root] INFO: 신규 페이지 저장
2026-08-18 00:57:05 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/about-us/careers/ 29 29
2026-08-18 00:57:05 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%9D%B8%EA%B3%B5%EC%A7%80%EB%8A%A5 29 29
2026-08-18 00:57:05 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-FTStatista/ 29 29
2026-08-18 00:57:17 [root] INFO: 신규 페이지 저장
2026-08-18 00:57:28 [root] INFO: 신규 페이지 저장
2026-08-18 00:57:39 [root] INFO: 신규 페이지 저장
2026-08-18 00:57:44 [scrapy.extensions.logstats] INFO: Crawled 32 pages (at 4 pages/min), scraped 32 items (at 4 items/min)
2026-08-18 00:57:46 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=Textanding 32 32
2026-08-18 00:57:56 [root] INFO: 신규 페이지 저장
2026-08-18 00:57:57 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=highest 33 33
2026-08-18 00:57:57 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=OCRNLP 33 33
2026-08-18 00:57:57 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=APAC 33 33
2026-08-18 00:58:08 [root] INFO: 신규 페이지 저장
2026-08-18 00:58:21 [root] INFO: 신규 페이지 저장
2026-08-18 00:58:31 [root] INFO: 신규 페이지 저장
2026-08-18 00:58:38 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=emerging 36 36
2026-08-18 00:58:48 [root] INFO: 신규 페이지 저장
2026-08-18 00:58:48 [scrapy.extensions.logstats] INFO: Crawled 37 pages (at 5 pages/min), scraped 37 items (at 5 items/min)
2026-08-18 00:58:48 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=growth 37 37
2026-08-18 00:58:49 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%95%A4%EC%84%9C%EB%A7%81AI 37 37
2026-08-18 00:58:49 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-OCRNLPTextanding/ 37 37
2026-08-18 00:58:59 [root] INFO: 신규 페이지 저장
2026-08-18 00:59:11 [root] INFO: 신규 페이지 저장
2026-08-18 00:59:23 [root] INFO: 신규 페이지 저장
2026-08-18 00:59:29 [sb-spider-v1] INFO: request callback: 500, request_sub_id: 1, processed_item_count: 40
2026-08-18 00:59:29 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%ED%85%8D%EC%8A%A4%ED%83%A0%EB%94%A9 40 40
2026-08-18 00:59:39 [root] INFO: 신규 페이지 저장
2026-08-18 00:59:40 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EA%B8%B0%EA%B3%84%EB%8F%85%ED%95%B4 41 41
2026-08-18 00:59:40 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-TTAmrctestcertification/ 41 41
2026-08-18 00:59:40 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=TTA 41 41
2026-08-18 00:59:51 [root] INFO: 신규 페이지 저장
2026-08-18 00:59:51 [scrapy.extensions.logstats] INFO: Crawled 44 pages (at 7 pages/min), scraped 42 items (at 5 items/min)
2026-08-18 01:00:03 [root] INFO: 신규 페이지 저장
2026-08-18 01:00:15 [root] INFO: 신규 페이지 저장
2026-08-18 01:00:22 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%ED%92%88%EC%A7%88%EC%84%B1%EB%8A%A5%ED%8F%89%EA%B0%80%ED%86%B5%EA%B3%BC 44 44
2026-08-18 01:00:31 [root] INFO: 신규 페이지 저장
2026-08-18 01:00:32 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=QA%ED%94%8C%EB%9E%AB%ED%8F%BC 45 45
2026-08-18 01:00:32 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%9D%B8%EA%B3%B5%EC%A7%80%EB%8A%A5%EC%B1%97%EB%B4%87 45 45
2026-08-18 01:00:32 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%9C%A0%EC%9D%BC%ED%86%B5%EA%B3%BC 45 45
2026-08-18 01:00:43 [root] INFO: 신규 페이지 저장
2026-08-18 01:00:54 [root] INFO: 신규 페이지 저장
2026-08-18 01:00:54 [scrapy.extensions.logstats] INFO: Crawled 48 pages (at 4 pages/min), scraped 47 items (at 5 items/min)
2026-08-18 01:01:05 [root] INFO: 신규 페이지 저장
2026-08-18 01:01:12 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-hackarthon/ 48 48
2026-08-18 01:01:12 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%ED%88%AC%EC%9E%90 48 48
2026-08-18 01:01:12 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%ED%95%B4%EC%BB%A4%ED%86%A4 48 48
2026-08-18 01:01:23 [root] INFO: 신규 페이지 저장
2026-08-18 01:01:34 [root] INFO: 신규 페이지 저장
2026-08-18 01:01:46 [root] INFO: 신규 페이지 저장
2026-08-18 01:01:46 [scrapy.extensions.logstats] INFO: Crawled 51 pages (at 3 pages/min), scraped 51 items (at 4 items/min)
2026-08-18 01:01:47 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-investmentA/ 51 51
2026-08-18 01:01:58 [root] INFO: 신규 페이지 저장
2026-08-18 01:02:04 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%ED%95%B4%EC%99%B8%EC%A7%84%EC%B6%9C 52 52
2026-08-18 01:02:13 [root] INFO: 신규 페이지 저장
2026-08-18 01:02:14 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-MWCLA2021/ 53 53
2026-08-18 01:02:14 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%8A%A4%ED%83%80%ED%8A%B8%EC%97%85 53 53
2026-08-18 01:02:14 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/?page=1 53 53
2026-08-18 01:02:25 [root] INFO: 신규 페이지 저장
2026-08-18 01:02:38 [root] INFO: 신규 페이지 저장
2026-08-18 01:02:48 [root] INFO: 중복 문서. 다른 URL 발견. additional_url 추가
2026-08-18 01:02:48 [scrapy.extensions.logstats] INFO: Crawled 56 pages (at 5 pages/min), scraped 56 items (at 5 items/min)
2026-08-18 01:02:55 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/?page=2 56 56
2026-08-18 01:03:05 [root] INFO: 신규 페이지 저장
2026-08-18 01:03:06 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/?page=4 57 57
2026-08-18 01:03:06 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/?page=3 57 57
2026-08-18 01:03:06 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/?page=5 57 57
2026-08-18 01:03:18 [root] INFO: 신규 페이지 저장
2026-08-18 01:03:29 [root] INFO: 신규 페이지 저장
2026-08-18 01:03:41 [root] INFO: 신규 페이지 저장
2026-08-18 01:03:44 [scrapy.extensions.logstats] INFO: Crawled 60 pages (at 4 pages/min), scraped 60 items (at 4 items/min)
2026-08-18 01:03:48 [sb-spider-v1] INFO: request callback: 500, request_sub_id: 2, processed_item_count: 60
2026-08-18 01:03:48 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/?page=6 60 60
2026-08-18 01:03:59 [root] INFO: 신규 페이지 저장
2026-08-18 01:03:59 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=solution 61 61
2026-08-18 01:03:59 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=award 61 61
2026-08-18 01:03:59 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=article 61 61
2026-08-18 01:04:10 [root] INFO: 신규 페이지 저장
2026-08-18 01:04:22 [root] INFO: 신규 페이지 저장
2026-08-18 01:04:34 [root] INFO: 신규 페이지 저장
2026-08-18 01:04:40 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=shipbuilding 64 64
2026-08-18 01:04:49 [root] INFO: 신규 페이지 저장
2026-08-18 01:04:49 [scrapy.extensions.logstats] INFO: Crawled 65 pages (at 5 pages/min), scraped 65 items (at 5 items/min)
2026-08-18 01:04:50 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=deep%20learning 65 65
2026-08-18 01:04:50 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=ai 65 65
2026-08-18 01:04:50 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=engineering 65 65
2026-08-18 01:05:01 [root] INFO: 신규 페이지 저장
2026-08-18 01:05:05 [root] INFO: 페이지 변경. update all_fields
2026-08-18 01:05:16 [root] INFO: 신규 페이지 저장
2026-08-18 01:05:23 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=event 68 68
2026-08-18 01:05:33 [root] INFO: 신규 페이지 저장
2026-08-18 01:05:34 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=Fintech 69 69
2026-08-18 01:05:34 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=program 69 69
2026-08-18 01:05:34 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=hr 69 69
2026-08-18 01:05:45 [root] INFO: 신규 페이지 저장
2026-08-18 01:05:45 [scrapy.extensions.logstats] INFO: Crawled 72 pages (at 7 pages/min), scraped 70 items (at 5 items/min)
2026-08-18 01:05:56 [root] INFO: 신규 페이지 저장
2026-08-18 01:06:09 [root] INFO: 신규 페이지 저장
2026-08-18 01:06:17 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=partners 72 72
2026-08-18 01:06:17 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=Research 72 72
2026-08-18 01:06:17 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EC%9D%B8%ED%84%B0%EB%B7%B0 72 72
2026-08-18 01:06:28 [root] INFO: 신규 페이지 저장
2026-08-18 01:06:40 [root] INFO: 신규 페이지 저장
2026-08-18 01:06:51 [root] INFO: 신규 페이지 저장
2026-08-18 01:06:51 [scrapy.extensions.logstats] INFO: Crawled 75 pages (at 3 pages/min), scraped 75 items (at 5 items/min)
2026-08-18 01:06:51 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=tv 75 75
2026-08-18 01:07:02 [root] INFO: 신규 페이지 저장
2026-08-18 01:07:08 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=Exhibition 76 76
2026-08-18 01:07:20 [root] INFO: 신규 페이지 저장
2026-08-18 01:07:21 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EB%A0%88%EA%B7%B8%ED%85%8C%ED%81%AC 77 77
2026-08-18 01:07:21 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%ED%95%80%ED%85%8C%ED%81%AC 77 77
2026-08-18 01:07:21 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=%EB%84%A5%EC%8A%A4%ED%8A%B8%EB%9D%BC%EC%9D%B4%EC%A6%88 77 77
2026-08-18 01:07:32 [root] INFO: 신규 페이지 저장
2026-08-18 01:07:45 [root] INFO: 신규 페이지 저장
2026-08-18 01:07:45 [scrapy.extensions.logstats] INFO: Crawled 80 pages (at 5 pages/min), scraped 79 items (at 4 items/min)
2026-08-18 01:07:56 [root] INFO: 신규 페이지 저장
2026-08-18 01:08:02 [sb-spider-v1] INFO: request callback: 500, request_sub_id: 3, processed_item_count: 80
2026-08-18 01:08:02 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AI%EC%83%81%EB%8B%B4%EC%9B%90 80 80
2026-08-18 01:08:13 [root] INFO: 신규 페이지 저장
2026-08-18 01:08:14 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/tags/?tag=AI%EC%BB%A8%ED%83%9D%EC%84%BC%ED%84%B0 81 81
2026-08-18 01:08:14 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-regtech/ 81 81
2026-08-18 01:08:14 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-WIS/ 81 81
2026-08-18 01:08:26 [root] INFO: 신규 페이지 저장
2026-08-18 01:08:38 [root] INFO: 신규 페이지 저장
2026-08-18 01:08:48 [root] INFO: 신규 페이지 저장
2026-08-18 01:08:48 [scrapy.extensions.logstats] INFO: Crawled 84 pages (at 4 pages/min), scraped 84 items (at 5 items/min)
2026-08-18 01:08:55 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-Nextrise/ 84 84
2026-08-18 01:09:05 [root] INFO: 신규 페이지 저장
2026-08-18 01:09:05 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-digitalNewDeal-Govproject/ 85 85
2026-08-18 01:09:06 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-Swingchat/ 85 85
2026-08-18 01:09:06 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/about-us/history/ 85 85
2026-08-18 01:09:15 [root] INFO: 신규 페이지 저장
2026-08-18 01:09:24 [root] INFO: 신규 페이지 저장
2026-08-18 01:09:33 [root] INFO: 신규 페이지 저장
2026-08-18 01:09:39 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-exhibition-AIEXPO-KOREA/ 88 88
2026-08-18 01:09:49 [root] INFO: 신규 페이지 저장
2026-08-18 01:09:49 [scrapy.extensions.logstats] INFO: Crawled 89 pages (at 5 pages/min), scraped 89 items (at 5 items/min)
2026-08-18 01:09:49 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-has-thesis-adopted-acl/ 89 89
2026-08-18 01:09:50 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/recruiting-kbs-2020-job-fair/ 89 89
2026-08-18 01:09:50 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-korea-fintech-week-2020/ 89 89
2026-08-18 01:10:00 [root] INFO: 신규 페이지 저장
2026-08-18 01:10:09 [root] INFO: 신규 페이지 저장
2026-08-18 01:10:19 [root] INFO: 신규 페이지 저장
2026-08-18 01:10:25 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-soft-wave-2019/ 92 92
2026-08-18 01:10:34 [root] INFO: 신규 페이지 저장
2026-08-18 01:10:35 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/successfully-completed-tips-program/ 93 93
2026-08-18 01:10:35 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/general-income-tax-consultation-ai-chatbot/ 93 93
2026-08-18 01:10:35 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/startup-intellectual-property-voucher/ 93 93
2026-08-18 01:10:44 [root] INFO: 신규 페이지 저장
2026-08-18 01:10:44 [scrapy.extensions.logstats] INFO: Crawled 96 pages (at 7 pages/min), scraped 94 items (at 5 items/min)
2026-08-18 01:10:54 [root] INFO: 신규 페이지 저장
2026-08-18 01:11:04 [root] INFO: 신규 페이지 저장
2026-08-18 01:11:11 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-participating-zero-one-day-2019/ 96 96
2026-08-18 01:11:21 [root] INFO: 신규 페이지 저장
2026-08-18 01:11:21 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-domain-independent-qa-model/ 97 97
2026-08-18 01:11:21 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-selected-samsungs-c-lab/ 97 97
2026-08-18 01:11:21 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-selected-kotra-global-jump-300/ 97 97
2026-08-18 01:11:32 [root] INFO: 신규 페이지 저장
2026-08-18 01:11:42 [root] INFO: 신규 페이지 저장
2026-08-18 01:11:52 [root] INFO: 신규 페이지 저장
2026-08-18 01:11:52 [scrapy.extensions.logstats] INFO: Crawled 100 pages (at 4 pages/min), scraped 100 items (at 6 items/min)
2026-08-18 01:12:00 [sb-spider-v1] INFO: request callback: 500, request_sub_id: 4, processed_item_count: 100
2026-08-18 01:12:00 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-venture-startup-summit-2019/ 100 100
2026-08-18 01:12:00 [sb-spider-v1] INFO: request callback: 500, request_sub_id: 5, processed_item_count: 100
2026-08-18 01:12:00 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-shinhan-futures-lab/ 100 100
2026-08-18 01:12:09 [root] INFO: 신규 페이지 저장
2026-08-18 01:12:20 [root] INFO: 신규 페이지 저장
2026-08-18 01:12:20 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-at-tomato-tv-show-startup-korea/ 102 102
2026-08-18 01:12:30 [root] INFO: 신규 페이지 저장
2026-08-18 01:12:30 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/kb-financial-groups-startup-incubating-program-kb-starters/ 103 103
2026-08-18 01:12:39 [root] INFO: 신규 페이지 저장
2026-08-18 01:12:44 [scrapy.extensions.logstats] INFO: Crawled 104 pages (at 4 pages/min), scraped 104 items (at 4 items/min)
2026-08-18 01:12:46 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-participate-ifa-2019/ 104 104
2026-08-18 01:12:55 [root] INFO: 신규 페이지 저장
2026-08-18 01:12:55 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/first-penguin-korea-credit-guarantee-fund/ 105 105
2026-08-18 01:12:55 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/liberation-algorithm/ 105 105
2026-08-18 01:12:55 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-aws-techshift-2019/ 105 105
2026-08-18 01:13:06 [root] INFO: 신규 페이지 저장
2026-08-18 01:13:18 [root] INFO: 신규 페이지 저장
2026-08-18 01:13:28 [root] INFO: 신규 페이지 저장
2026-08-18 01:13:35 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-ces-2020/ 108 108
2026-08-18 01:13:44 [root] INFO: 신규 페이지 저장
2026-08-18 01:13:44 [scrapy.extensions.logstats] INFO: Crawled 109 pages (at 5 pages/min), scraped 109 items (at 5 items/min)
2026-08-18 01:13:45 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/award-from-ministry-of-sme/ 109 109
2026-08-18 01:13:45 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/mou-singapore-intage/ 109 109
2026-08-18 01:13:45 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/introducing-jerry-qa-technical-team-leader/ 109 109
2026-08-18 01:13:55 [root] INFO: 신규 페이지 저장
2026-08-18 01:14:04 [root] INFO: 신규 페이지 저장
2026-08-18 01:14:14 [root] INFO: 신규 페이지 저장
2026-08-18 01:14:20 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/winners-kgcci-5th-innovation-awards/ 112 112
2026-08-18 01:14:30 [root] INFO: 신규 페이지 저장
2026-08-18 01:14:31 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/award-global-saas-development-project/ 113 113
2026-08-18 01:14:31 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-european-chatbot-events-2019/ 113 113
2026-08-18 01:14:31 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/search-system-improvements-ai/ 113 113
2026-08-18 01:14:40 [root] INFO: 신규 페이지 저장
2026-08-18 01:14:49 [root] INFO: 신규 페이지 저장
2026-08-18 01:14:49 [scrapy.extensions.logstats] INFO: Crawled 116 pages (at 7 pages/min), scraped 115 items (at 6 items/min)
2026-08-18 01:14:58 [root] INFO: 신규 페이지 저장
2026-08-18 01:15:03 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-participates-london-tech-week-2019/ 116 116
2026-08-18 01:15:13 [root] INFO: 신규 페이지 저장
2026-08-18 01:15:13 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/korea-lenovo-and-ai-start-up-42maru-partnership/ 117 117
2026-08-18 01:15:13 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/tech-startup-42maru-eyes-foreign-market-deep-learning-solution/ 117 117
2026-08-18 01:15:13 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/dsme-develops-ai-system-deal-shipowners/ 117 117
2026-08-18 01:15:22 [root] INFO: 신규 페이지 저장
2026-08-18 01:15:32 [root] INFO: 신규 페이지 저장
2026-08-18 01:15:41 [root] INFO: 신규 페이지 저장
2026-08-18 01:15:44 [scrapy.extensions.logstats] INFO: Crawled 120 pages (at 4 pages/min), scraped 120 items (at 5 items/min)
2026-08-18 01:15:46 [sb-spider-v1] INFO: request callback: 500, request_sub_id: 6, processed_item_count: 120
2026-08-18 01:15:46 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/kr/blog/42maru-wins-aws-ai-startup-competition/ 120 120
2026-08-18 01:15:55 [root] INFO: 신규 페이지 저장
2026-08-18 01:15:55 [scrapy.core.engine] INFO: Closing spider (finished)
2026-08-18 01:15:55 [sb-spider-v1] INFO: Spider 'sb-spider-v1' closed. Reason : finished
2026-08-18 01:15:55 [sb-spider-v1] INFO: spider_closed function called
2026-08-18 01:15:55 [sb-spider-v1] INFO: reason - finished
2026-08-18 01:15:55 [sb-spider-v1] INFO: scraped_item_count - 121
2026-08-18 01:15:55 [sb-spider-v1] INFO: processed_item_count - 121
2026-08-18 01:15:56 [sb-spider-v1] INFO: request callback (finished): 500, request_sub_id: 7, processed_item_count: 121
2026-08-18 01:15:56 [sb-spider-v1] INFO: Message sent successfully to Slack!
2026-08-18 01:16:01 [sb-spider-v1] INFO: Update Not Crawled Page Status - D
2026-08-18 01:16:01 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 35108,
 'downloader/request_count': 121,
 'downloader/request_method_count/GET': 121,
 'downloader/response_bytes': 3285241,
 'downloader/response_count': 121,
 'downloader/response_status_count/200': 121,
 'dupefilter/filtered': 1640,
 'elapsed_time_seconds': 1511.592623,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2026, 8, 18, 1, 15, 55, 994338),
 'item_scraped_count': 121,
 'log_count/INFO': 300,
 'memusage/max': 334376960,
 'memusage/startup': 109285376,
 'pipeline/items_processed': 121,
 'playwright/context_count': 1,
 'playwright/context_count/max_concurrent': 1,
 'playwright/context_count/non_persistent': 1,
 'playwright/page_count': 121,
 'playwright/page_count/max_concurrent': 4,
 'playwright/request_count': 5110,
 'playwright/request_count/method/GET': 4418,
 'playwright/request_count/method/POST': 692,
 'playwright/request_count/navigation': 123,
 'playwright/request_count/resource_type/document': 123,
 'playwright/request_count/resource_type/fetch': 194,
 'playwright/request_count/resource_type/image': 1143,
 'playwright/request_count/resource_type/ping': 377,
 'playwright/request_count/resource_type/script': 2419,
 'playwright/request_count/resource_type/stylesheet': 605,
 'playwright/request_count/resource_type/xhr': 249,
 'playwright/response_count': 5095,
 'playwright/response_count/method/GET': 4417,
 'playwright/response_count/method/POST': 678,
 'playwright/response_count/resource_type/document': 123,
 'playwright/response_count/resource_type/fetch': 185,
 'playwright/response_count/resource_type/image': 1143,
 'playwright/response_count/resource_type/ping': 372,
 'playwright/response_count/resource_type/script': 2419,
 'playwright/response_count/resource_type/stylesheet': 605,
 'playwright/response_count/resource_type/xhr': 248,
 'request_depth_max': 4,
 'response_received_count': 121,
 'scheduler/dequeued': 121,
 'scheduler/dequeued/memory': 121,
 'scheduler/enqueued': 121,
 'scheduler/enqueued/memory': 121,
 'start_time': datetime.datetime(2026, 8, 18, 0, 50, 44, 401715)}
2026-08-18 01:16:01 [scrapy.core.engine] INFO: Spider closed (finished)
2026-08-18 01:16:01 [scrapy-playwright] INFO: Closing download handler
2026-08-18 01:16:01 [scrapy-playwright] INFO: Closing download handler
2026-08-18 01:16:01 [scrapy-playwright] INFO: Closing browser