2026-08-18 00:47:39 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: sitebunny) 2026-08-18 00:47:39 [scrapy.utils.log] INFO: Versions: lxml 4.9.3.0, libxml2 2.10.3, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.2, Twisted 22.10.0, Python 3.11.13 (main, Jul 22 2025, 01:43:09) [GCC 10.2.1 20210110], pyOpenSSL 23.2.0 (OpenSSL 3.1.2 1 Aug 2023), cryptography 41.0.3, Platform Linux-5.15.0-141-generic-x86_64-with-glibc2.31 2026-08-18 00:47:39 [scrapy.addons] INFO: Enabled addons: [] 2026-08-18 00:47:39 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'sitebunny', 'CLOSESPIDER_PAGECOUNT': '2001', 'CONCURRENT_REQUESTS': '4', 'COOKIES_ENABLED': False, 'DEPTH_LIMIT': '10', 'DEPTH_PRIORITY': 1, 'DOWNLOAD_DELAY': 1, 'FEED_EXPORT_ENCODING': 'utf-8', 'LOG_FILE': '/data/webcrawl/scrapyd.dev/logs/sitebunny/sb-spider-v1/6845bcf09a9e11f1ba910242ac130002.log', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'sitebunny.spiders', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'ROBOTSTXT_OBEY': 'False', 'SCHEDULER_DISK_QUEUE': 'scrapy.squeues.PickleFifoDiskQueue', 'SCHEDULER_MEMORY_QUEUE': 'scrapy.squeues.FifoMemoryQueue', 'SPIDER_MODULES': ['sitebunny.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'} 2026-08-18 00:47:39 [scrapy.extensions.telnet] INFO: Telnet Password: 8bd68031a2124664 2026-08-18 00:47:39 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.closespider.CloseSpider', 'scrapy.extensions.logstats.LogStats'] 2026-08-18 00:47:39 [sb-spider-v1] INFO: {'allow': 'https?://42maru.ai/en', 'deny': 'signin|sign-in|Signin|SignIn|signIn|login|log-in|Login|LogIn|logIn|signup|sign-up|Signup|SignUp|signUp|findid|find-id|FindId|FindID|findId|findID|findpw|find-pw|FindPw|FindPW|findPw|findPW|findpassword|find-password|FindPassword|findPassword|findaccount|find-account|FindAccount|findAccount|/idpw|forgot-password|change-password|mode=download|downloadFile|/file/download|download.php|/download|/bbs/down.php|attachFile/download|getVcard.|dispBoardTagList|.(jpg|pdf|hwp|xls|xlsx|doc|docx|zip)$'} 2026-08-18 00:47:39 [sb-spider-v1] INFO: {'link_extractor': , 'callback': 'parse_item', 'follow': True, 'process_request': 'set_playwright', 'errback': 'errback'} 2026-08-18 00:47:39 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2026-08-18 00:47:39 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2026-08-18 00:47:39 [scrapy.middleware] INFO: Enabled item pipelines: ['sitebunny.pipelines.SitebunnyPipeline'] 2026-08-18 00:47:39 [scrapy.core.engine] INFO: Spider opened 2026-08-18 00:47:39 [scrapy.extensions.closespider] INFO: Spider will stop when no items are produced after 300 seconds. 2026-08-18 00:47:39 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2026-08-18 00:47:39 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2026-08-18 00:47:39 [scrapy-playwright] INFO: Starting download handler 2026-08-18 00:47:39 [scrapy-playwright] INFO: Starting download handler 2026-08-18 00:47:44 [sb-spider-v1] INFO: Message sent successfully to Slack! 2026-08-18 00:47:44 [scrapy-playwright] INFO: Launching browser chromium 2026-08-18 00:47:44 [scrapy-playwright] INFO: Browser chromium launched 2026-08-18 00:47:52 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/ None None 2026-08-18 00:48:03 [root] INFO: 신규 페이지 저장 2026-08-18 00:48:11 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/ 1 1 2026-08-18 00:48:11 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/contact/ 1 1 2026-08-18 00:48:21 [root] INFO: 신규 페이지 저장 2026-08-18 00:48:22 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/ 2 2 2026-08-18 00:48:22 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/ 2 2 2026-08-18 00:48:34 [root] INFO: 신규 페이지 저장 2026-08-18 00:48:40 [root] INFO: 기수집. 미업데이트. update last_crawled_at 2026-08-18 00:48:40 [scrapy.extensions.logstats] INFO: Crawled 5 pages (at 5 pages/min), scraped 4 items (at 4 items/min) 2026-08-18 00:48:46 [root] INFO: 기수집. 미업데이트. update last_crawled_at 2026-08-18 00:48:49 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/about-us/ 5 5 2026-08-18 00:48:59 [root] INFO: 신규 페이지 저장 2026-08-18 00:49:00 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/about-us/partners/ 6 6 2026-08-18 00:49:00 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/about-us/our-team/ 6 6 2026-08-18 00:49:09 [root] INFO: 신규 페이지 저장 2026-08-18 00:49:17 [root] INFO: 신규 페이지 저장 2026-08-18 00:49:18 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/about-us/investors/ 8 8 2026-08-18 00:49:28 [root] INFO: 신규 페이지 저장 2026-08-18 00:49:33 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/about-us/history/ 9 9 2026-08-18 00:49:45 [root] INFO: 신규 페이지 저장 2026-08-18 00:49:45 [scrapy.extensions.logstats] INFO: Crawled 10 pages (at 5 pages/min), scraped 10 items (at 6 items/min) 2026-08-18 00:49:47 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 0, processed_item_count: 10 2026-08-18 00:49:47 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ 10 10 2026-08-18 00:49:48 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 1, processed_item_count: 10 2026-08-18 00:49:48 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-large-enterprise/ 10 10 2026-08-18 00:49:49 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 2, processed_item_count: 10 2026-08-18 00:49:49 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-based-chatbot/ 10 10 2026-08-18 00:50:02 [root] INFO: 신규 페이지 저장 2026-08-18 00:50:12 [root] INFO: 신규 페이지 저장 2026-08-18 00:50:22 [root] INFO: 신규 페이지 저장 2026-08-18 00:50:26 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-smart-devices/ 13 13 2026-08-18 00:50:38 [root] INFO: 신규 페이지 저장 2026-08-18 00:50:39 [scrapy.extensions.logstats] INFO: Crawled 14 pages (at 4 pages/min), scraped 14 items (at 4 items/min) 2026-08-18 00:50:39 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/ 14 14 2026-08-18 00:50:39 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/technology/ 14 14 2026-08-18 00:50:39 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/ai-document-search-and-comparison/ 14 14 2026-08-18 00:50:50 [root] INFO: 신규 페이지 저장 2026-08-18 00:51:01 [root] INFO: 신규 페이지 저장 2026-08-18 00:51:12 [root] INFO: 신규 페이지 저장 2026-08-18 00:51:17 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/customer-ai-based-chatbot/ 17 17 2026-08-18 00:51:29 [root] INFO: 신규 페이지 저장 2026-08-18 00:51:29 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/standard-specification-comparison/ 18 18 2026-08-18 00:51:29 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/contract-comprehension-and-comparison/ 18 18 2026-08-18 00:51:43 [root] INFO: 신규 페이지 저장 2026-08-18 00:51:43 [scrapy.extensions.logstats] INFO: Crawled 20 pages (at 6 pages/min), scraped 19 items (at 5 items/min) 2026-08-18 00:51:55 [root] INFO: 신규 페이지 저장 2026-08-18 00:51:57 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 3, processed_item_count: 20 2026-08-18 00:51:57 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/ 20 20 2026-08-18 00:52:03 [root] INFO: 기수집. 미업데이트. update last_crawled_at 2026-08-18 00:52:09 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/ 21 21 2026-08-18 00:52:09 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-has-thesis-adopted-acl/ 21 21 2026-08-18 00:52:09 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=Research 21 21 2026-08-18 00:52:14 [root] INFO: 기수집. 미업데이트. update last_crawled_at 2026-08-18 00:52:25 [root] INFO: 신규 페이지 저장 2026-08-18 00:52:38 [root] INFO: 신규 페이지 저장 2026-08-18 00:52:38 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/kdb-bank-selects-42maru-innovative-startup/ 24 24 2026-08-18 00:52:39 [scrapy.extensions.logstats] INFO: Crawled 25 pages (at 5 pages/min), scraped 24 items (at 5 items/min) 2026-08-18 00:52:52 [root] INFO: 신규 페이지 저장 2026-08-18 00:52:57 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=program 25 25 2026-08-18 00:52:57 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=tv 25 25 2026-08-18 00:52:57 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/recruiting-kbs-2020-job-fair/ 25 25 2026-08-18 00:53:09 [root] INFO: 신규 페이지 저장 2026-08-18 00:53:21 [root] INFO: 신규 페이지 저장 2026-08-18 00:53:31 [root] INFO: 신규 페이지 저장 2026-08-18 00:53:32 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=hr 28 28 2026-08-18 00:53:44 [root] INFO: 신규 페이지 저장 2026-08-18 00:53:44 [scrapy.extensions.logstats] INFO: Crawled 29 pages (at 4 pages/min), scraped 29 items (at 5 items/min) 2026-08-18 00:53:50 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-korea-fintech-week-2020/ 29 29 2026-08-18 00:54:01 [root] INFO: 신규 페이지 저장 2026-08-18 00:54:02 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 4, processed_item_count: 30 2026-08-18 00:54:02 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/legal/ 30 30 2026-08-18 00:54:14 [root] INFO: 신규 페이지 저장 2026-08-18 00:54:14 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/about-us/careers/ 31 31 2026-08-18 00:54:14 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=event 31 31 2026-08-18 00:54:14 [sb-spider-v1] INFO: 파싱 제외. response status 404 2026-08-18 00:54:26 [root] INFO: 신규 페이지 저장 2026-08-18 00:54:31 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/general-income-tax-consultation-ai-chatbot/ 32 32 2026-08-18 00:54:43 [root] INFO: 신규 페이지 저장 2026-08-18 00:54:43 [scrapy.extensions.logstats] INFO: Crawled 34 pages (at 5 pages/min), scraped 33 items (at 4 items/min) 2026-08-18 00:54:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=solution 33 33 2026-08-18 00:54:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-selected-kotra-global-jump-300/ 33 33 2026-08-18 00:54:44 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/startup-intellectual-property-voucher/ 33 33 2026-08-18 00:54:55 [root] INFO: 신규 페이지 저장 2026-08-18 00:55:05 [root] INFO: 신규 페이지 저장 2026-08-18 00:55:15 [root] INFO: 신규 페이지 저장 2026-08-18 00:55:21 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/kb-financial-groups-startup-incubating-program-kb-starters/ 36 36 2026-08-18 00:55:33 [root] INFO: 신규 페이지 저장 2026-08-18 00:55:33 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=Fintech 37 37 2026-08-18 00:55:33 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/first-penguin-korea-credit-guarantee-fund/ 37 37 2026-08-18 00:55:33 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/?page=1 37 37 2026-08-18 00:55:44 [root] INFO: 신규 페이지 저장 2026-08-18 00:55:44 [scrapy.extensions.logstats] INFO: Crawled 41 pages (at 7 pages/min), scraped 38 items (at 5 items/min) 2026-08-18 00:55:56 [root] INFO: 신규 페이지 저장 2026-08-18 00:56:08 [root] INFO: 중복 문서. 다른 URL 발견. additional_url 추가 2026-08-18 00:56:14 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 5, processed_item_count: 40 2026-08-18 00:56:14 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/?page=2 40 40 2026-08-18 00:56:25 [root] INFO: 신규 페이지 저장 2026-08-18 00:56:25 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=article 41 41 2026-08-18 00:56:25 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/?page=3 41 41 2026-08-18 00:56:25 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/?page=4 41 41 2026-08-18 00:56:37 [root] INFO: 신규 페이지 저장 2026-08-18 00:56:49 [root] INFO: 신규 페이지 저장 2026-08-18 00:56:49 [scrapy.extensions.logstats] INFO: Crawled 45 pages (at 4 pages/min), scraped 43 items (at 5 items/min) 2026-08-18 00:57:00 [root] INFO: 신규 페이지 저장 2026-08-18 00:57:05 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=award 44 44 2026-08-18 00:57:16 [root] INFO: 신규 페이지 저장 2026-08-18 00:57:17 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=shipbuilding 45 45 2026-08-18 00:57:17 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=engineering 45 45 2026-08-18 00:57:17 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=partners 45 45 2026-08-18 00:57:28 [root] INFO: 신규 페이지 저장 2026-08-18 00:57:40 [root] INFO: 신규 페이지 저장 2026-08-18 00:57:40 [scrapy.extensions.logstats] INFO: Crawled 49 pages (at 4 pages/min), scraped 47 items (at 4 items/min) 2026-08-18 00:57:51 [root] INFO: 신규 페이지 저장 2026-08-18 00:57:57 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=ai 48 48 2026-08-18 00:58:08 [root] INFO: 신규 페이지 저장 2026-08-18 00:58:08 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/tags/?tag=deep%20learning 49 49 2026-08-18 00:58:08 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-large-enterprise/document-comprehension/ 49 49 2026-08-18 00:58:08 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/award-global-saas-development-project/ 49 49 2026-08-18 00:58:20 [root] INFO: 신규 페이지 저장 2026-08-18 00:58:31 [root] INFO: 신규 페이지 저장 2026-08-18 00:58:41 [root] INFO: 신규 페이지 저장 2026-08-18 00:58:41 [scrapy.extensions.logstats] INFO: Crawled 53 pages (at 4 pages/min), scraped 52 items (at 5 items/min) 2026-08-18 00:58:46 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/award-from-ministry-of-sme/ 52 52 2026-08-18 00:58:58 [root] INFO: 신규 페이지 저장 2026-08-18 00:58:59 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/winners-kgcci-5th-innovation-awards/ 53 53 2026-08-18 00:58:59 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-participating-zero-one-day-2019/ 53 53 2026-08-18 00:58:59 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-selected-samsungs-c-lab/ 53 53 2026-08-18 00:59:11 [root] INFO: 신규 페이지 저장 2026-08-18 00:59:23 [root] INFO: 신규 페이지 저장 2026-08-18 00:59:33 [root] INFO: 신규 페이지 저장 2026-08-18 00:59:39 [scrapy.extensions.logstats] INFO: Crawled 58 pages (at 5 pages/min), scraped 56 items (at 4 items/min) 2026-08-18 00:59:39 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-shinhan-futures-lab/ 56 56 2026-08-18 00:59:51 [root] INFO: 신규 페이지 저장 2026-08-18 00:59:52 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/mou-singapore-intage/ 57 57 2026-08-18 00:59:52 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/automatic-customer-answers/ 57 57 2026-08-18 00:59:52 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/liberation-algorithm/ 57 57 2026-08-18 01:00:04 [root] INFO: 신규 페이지 저장 2026-08-18 01:00:14 [root] INFO: 신규 페이지 저장 2026-08-18 01:00:24 [root] INFO: 신규 페이지 저장 2026-08-18 01:00:32 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 6, processed_item_count: 60 2026-08-18 01:00:32 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/voice-ai-customer-experiences-iot/ 60 60 2026-08-18 01:00:42 [root] INFO: 신규 페이지 저장 2026-08-18 01:00:42 [scrapy.extensions.logstats] INFO: Crawled 62 pages (at 4 pages/min), scraped 61 items (at 5 items/min) 2026-08-18 01:00:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/smart-device-voice-assistants/ 61 61 2026-08-18 01:00:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/cases/employee-support-ai-based-chatbot/ 61 61 2026-08-18 01:00:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/korea-lenovo-and-ai-start-up-42maru-partnership/ 61 61 2026-08-18 01:00:54 [root] INFO: 신규 페이지 저장 2026-08-18 01:01:05 [root] INFO: 신규 페이지 저장 2026-08-18 01:01:17 [root] INFO: 신규 페이지 저장 2026-08-18 01:01:22 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-domain-independent-qa-model/ 64 64 2026-08-18 01:01:33 [root] INFO: 신규 페이지 저장 2026-08-18 01:01:34 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-aws-techshift-2019/ 65 65 2026-08-18 01:01:34 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-european-chatbot-events-2019/ 65 65 2026-08-18 01:01:34 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-participates-london-tech-week-2019/ 65 65 2026-08-18 01:01:46 [root] INFO: 신규 페이지 저장 2026-08-18 01:01:46 [scrapy.extensions.logstats] INFO: Crawled 69 pages (at 7 pages/min), scraped 66 items (at 5 items/min) 2026-08-18 01:01:57 [root] INFO: 신규 페이지 저장 2026-08-18 01:02:08 [root] INFO: 신규 페이지 저장 2026-08-18 01:02:15 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/tech-startup-42maru-eyes-foreign-market-deep-learning-solution/ 68 68 2026-08-18 01:02:26 [root] INFO: 신규 페이지 저장 2026-08-18 01:02:27 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-large-enterprise/cognitive-search-engine/ 69 69 2026-08-18 01:02:27 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-large-enterprise/process-automation/ 69 69 2026-08-18 01:02:27 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-large-enterprise/specifications-matching/ 69 69 2026-08-18 01:02:37 [root] INFO: 신규 페이지 저장 2026-08-18 01:02:48 [root] INFO: 신규 페이지 저장 2026-08-18 01:02:48 [scrapy.extensions.logstats] INFO: Crawled 73 pages (at 4 pages/min), scraped 71 items (at 5 items/min) 2026-08-18 01:02:57 [root] INFO: 신규 페이지 저장 2026-08-18 01:03:02 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/solutions/ai-based-chatbot/ai-assistant-contact-centers/ 72 72 2026-08-18 01:03:15 [root] INFO: 신규 페이지 저장 2026-08-18 01:03:15 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=mrc 73 73 2026-08-18 01:03:15 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=engineering 73 73 2026-08-18 01:03:15 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=south%20korea 73 73 2026-08-18 01:03:29 [root] INFO: 신규 페이지 저장 2026-08-18 01:03:40 [root] INFO: 신규 페이지 저장 2026-08-18 01:03:40 [scrapy.extensions.logstats] INFO: Crawled 77 pages (at 4 pages/min), scraped 75 items (at 4 items/min) 2026-08-18 01:03:50 [root] INFO: 신규 페이지 저장 2026-08-18 01:03:56 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=nlu 76 76 2026-08-18 01:04:07 [root] INFO: 신규 페이지 저장 2026-08-18 01:04:08 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=search%20ir 77 77 2026-08-18 01:04:08 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=question%20answer 77 77 2026-08-18 01:04:08 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=semantic 77 77 2026-08-18 01:04:20 [root] INFO: 신규 페이지 저장 2026-08-18 01:04:33 [root] INFO: 신규 페이지 저장 2026-08-18 01:04:44 [root] INFO: 신규 페이지 저장 2026-08-18 01:04:44 [scrapy.extensions.logstats] INFO: Crawled 81 pages (at 4 pages/min), scraped 80 items (at 5 items/min) 2026-08-18 01:04:51 [sb-spider-v1] INFO: request callback: 200, request_sub_id: 7, processed_item_count: 80 2026-08-18 01:04:51 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=consumer%20electronics 80 80 2026-08-18 01:05:01 [root] INFO: 신규 페이지 저장 2026-08-18 01:05:01 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=automotive 81 81 2026-08-18 01:05:01 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=voice%20interaction 81 81 2026-08-18 01:05:02 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=paraphrasing 81 81 2026-08-18 01:05:12 [root] INFO: 신규 페이지 저장 2026-08-18 01:05:23 [root] INFO: 신규 페이지 저장 2026-08-18 01:05:34 [root] INFO: 신규 페이지 저장 2026-08-18 01:05:39 [scrapy.extensions.logstats] INFO: Crawled 85 pages (at 4 pages/min), scraped 84 items (at 4 items/min) 2026-08-18 01:05:40 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=singapore 84 84 2026-08-18 01:05:51 [root] INFO: 신규 페이지 저장 2026-08-18 01:05:52 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/successfully-completed-tips-program/ 85 85 2026-08-18 01:05:52 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/case-tags/?case_tags=hr 85 85 2026-08-18 01:05:52 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-at-tomato-tv-show-startup-korea/ 85 85 2026-08-18 01:06:03 [root] INFO: 신규 페이지 저장 2026-08-18 01:06:14 [root] INFO: 신규 페이지 저장 2026-08-18 01:06:25 [root] INFO: 신규 페이지 저장 2026-08-18 01:06:31 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-ces-2020/ 88 88 2026-08-18 01:06:43 [root] INFO: 신규 페이지 저장 2026-08-18 01:06:43 [scrapy.extensions.logstats] INFO: Crawled 90 pages (at 5 pages/min), scraped 89 items (at 5 items/min) 2026-08-18 01:06:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-venture-startup-summit-2019/ 89 89 2026-08-18 01:06:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-participate-ifa-2019/ 89 89 2026-08-18 01:06:43 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-soft-wave-2019/ 89 89 2026-08-18 01:06:57 [root] INFO: 신규 페이지 저장 2026-08-18 01:07:08 [root] INFO: 신규 페이지 저장 2026-08-18 01:07:20 [root] INFO: 신규 페이지 저장 2026-08-18 01:07:25 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/number-one-ai-company-korea/ 92 92 2026-08-18 01:07:35 [root] INFO: 신규 페이지 저장 2026-08-18 01:07:35 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/collaboration-between-42maru-and-cuckoo-electronics/ 93 93 2026-08-18 01:07:36 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/dsme-develops-ai-system-deal-shipowners/ 93 93 2026-08-18 01:07:36 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/search-system-improvements-ai/ 93 93 2026-08-18 01:07:46 [root] INFO: 신규 페이지 저장 2026-08-18 01:07:46 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 7 pages/min), scraped 94 items (at 5 items/min) 2026-08-18 01:07:58 [root] INFO: 신규 페이지 저장 2026-08-18 01:08:09 [root] INFO: 신규 페이지 저장 2026-08-18 01:08:15 [sb-spider-v1] INFO: Parse Item function called on https://42maru.ai/en/blog/42maru-wins-aws-ai-startup-competition/ 96 96 2026-08-18 01:08:27 [root] INFO: 신규 페이지 저장 2026-08-18 01:08:27 [scrapy.core.engine] INFO: Closing spider (finished) 2026-08-18 01:08:27 [sb-spider-v1] INFO: Spider 'sb-spider-v1' closed. Reason : finished 2026-08-18 01:08:27 [sb-spider-v1] INFO: spider_closed function called 2026-08-18 01:08:27 [sb-spider-v1] INFO: reason - finished 2026-08-18 01:08:27 [sb-spider-v1] INFO: scraped_item_count - 97 2026-08-18 01:08:27 [sb-spider-v1] INFO: processed_item_count - 97 2026-08-18 01:08:28 [sb-spider-v1] INFO: request callback (finished): 200, request_sub_id: 8, processed_item_count: 97 2026-08-18 01:08:29 [sb-spider-v1] INFO: Message sent successfully to Slack! 2026-08-18 01:08:35 [sb-spider-v1] INFO: Update Not Crawled Page Status - D 2026-08-18 01:08:35 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 27662, 'downloader/request_count': 98, 'downloader/request_method_count/GET': 98, 'downloader/response_bytes': 4098971, 'downloader/response_count': 98, 'downloader/response_status_count/200': 97, 'downloader/response_status_count/404': 1, 'dupefilter/filtered': 2337, 'elapsed_time_seconds': 1248.327902, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2026, 8, 18, 1, 8, 27, 713125), 'item_scraped_count': 97, 'log_count/INFO': 250, 'memusage/max': 322867200, 'memusage/startup': 109654016, 'pipeline/items_processed': 97, 'playwright/context_count': 1, 'playwright/context_count/max_concurrent': 1, 'playwright/context_count/non_persistent': 1, 'playwright/page_count': 98, 'playwright/page_count/max_concurrent': 4, 'playwright/request_count': 4655, 'playwright/request_count/method/GET': 4098, 'playwright/request_count/method/POST': 557, 'playwright/request_count/navigation': 103, 'playwright/request_count/resource_type/document': 103, 'playwright/request_count/resource_type/fetch': 152, 'playwright/request_count/resource_type/image': 1407, 'playwright/request_count/resource_type/ping': 304, 'playwright/request_count/resource_type/script': 1980, 'playwright/request_count/resource_type/stylesheet': 494, 'playwright/request_count/resource_type/xhr': 215, 'playwright/response_count': 4640, 'playwright/response_count/method/GET': 4096, 'playwright/response_count/method/POST': 544, 'playwright/response_count/resource_type/document': 103, 'playwright/response_count/resource_type/fetch': 146, 'playwright/response_count/resource_type/image': 1407, 'playwright/response_count/resource_type/ping': 299, 'playwright/response_count/resource_type/script': 1980, 'playwright/response_count/resource_type/stylesheet': 494, 'playwright/response_count/resource_type/xhr': 211, 'request_depth_max': 4, 'response_received_count': 98, 'scheduler/dequeued': 98, 'scheduler/dequeued/memory': 98, 'scheduler/enqueued': 98, 'scheduler/enqueued/memory': 98, 'start_time': datetime.datetime(2026, 8, 18, 0, 47, 39, 385223)} 2026-08-18 01:08:35 [scrapy.core.engine] INFO: Spider closed (finished) 2026-08-18 01:08:35 [scrapy-playwright] INFO: Closing download handler 2026-08-18 01:08:35 [scrapy-playwright] INFO: Closing download handler 2026-08-18 01:08:35 [scrapy-playwright] INFO: Closing browser