2026-08-18 00:40:49 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: sitebunny)
2026-08-18 00:40:49 [scrapy.utils.log] INFO: Versions: lxml 4.9.3.0, libxml2 2.10.3, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.2, Twisted 22.10.0, Python 3.11.13 (main, Jul 22 2025, 01:43:09) [GCC 10.2.1 20210110], pyOpenSSL 23.2.0 (OpenSSL 3.1.2 1 Aug 2023), cryptography 41.0.3, Platform Linux-5.15.0-141-generic-x86_64-with-glibc2.31
2026-08-18 00:40:49 [scrapy.addons] INFO: Enabled addons:
[]
2026-08-18 00:40:49 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'sitebunny',
 'CLOSESPIDER_ITEMCOUNT': '2',
 'CONCURRENT_ITEMS': '1',
 'CONCURRENT_REQUESTS': '1',
 'COOKIES_ENABLED': False,
 'DEPTH_LIMIT': '1',
 'DEPTH_PRIORITY': 1,
 'DOWNLOAD_DELAY': 1,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/data/webcrawl/scrapyd.dev/logs/sitebunny/sb-spider-v1/73efb7329a9d11f1ba910242ac130002.log',
 'LOG_LEVEL': 'INFO',
 'NEWSPIDER_MODULE': 'sitebunny.spiders',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'ROBOTSTXT_OBEY': 'False',
 'SCHEDULER_DISK_QUEUE': 'scrapy.squeues.PickleFifoDiskQueue',
 'SCHEDULER_MEMORY_QUEUE': 'scrapy.squeues.FifoMemoryQueue',
 'SPIDER_MODULES': ['sitebunny.spiders'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'}
2026-08-18 00:40:49 [scrapy.extensions.telnet] INFO: Telnet Password: 828727c784f1fa22
2026-08-18 00:40:49 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.closespider.CloseSpider',
 'scrapy.extensions.logstats.LogStats']
2026-08-18 00:40:49 [sb-spider-v1] INFO: {'allow': 'https?://www.42maru.ai', 'deny': 'signin|sign-in|Signin|SignIn|signIn|login|log-in|Login|LogIn|logIn|signup|sign-up|Signup|SignUp|signUp|findid|find-id|FindId|FindID|findId|findID|findpw|find-pw|FindPw|FindPW|findPw|findPW|findpassword|find-password|FindPassword|findPassword|findaccount|find-account|FindAccount|findAccount|/idpw|forgot-password|change-password|mode=download|downloadFile|/file/download|download.php|/download|/bbs/down.php|attachFile/download|getVcard.|dispBoardTagList|.(jpg|pdf|hwp|xls|xlsx|doc|docx|zip)$'}
2026-08-18 00:40:49 [sb-spider-v1] INFO: {'link_extractor': <scrapy.linkextractors.lxmlhtml.LxmlLinkExtractor object at 0x7f5a01179010>, 'callback': 'parse_item', 'follow': True, 'process_request': 'set_playwright', 'errback': 'errback'}
2026-08-18 00:40:49 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-08-18 00:40:49 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-08-18 00:40:49 [scrapy.middleware] INFO: Enabled item pipelines:
['sitebunny.pipelines.SitebunnyPipeline']
2026-08-18 00:40:49 [scrapy.core.engine] INFO: Spider opened
2026-08-18 00:40:49 [scrapy.extensions.closespider] INFO: Spider will stop when no items are produced after 300 seconds.
2026-08-18 00:40:49 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-08-18 00:40:49 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2026-08-18 00:40:49 [scrapy-playwright] INFO: Starting download handler
2026-08-18 00:40:49 [scrapy-playwright] INFO: Starting download handler
2026-08-18 00:40:55 [sb-spider-v1] INFO: Message sent successfully to Slack!
2026-08-18 00:40:55 [scrapy-playwright] INFO: Launching browser chromium
2026-08-18 00:40:55 [scrapy-playwright] INFO: Browser chromium launched
2026-08-18 00:41:02 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/en/ None None
2026-08-18 00:41:13 [root] INFO: 신규 페이지 저장
2026-08-18 00:41:20 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/en/ 1 1
2026-08-18 00:41:25 [root] INFO: 기수집. 미업데이트. update last_crawled_at
2026-08-18 00:41:25 [scrapy.core.engine] INFO: Closing spider (closespider_itemcount)
2026-08-18 00:41:31 [sb-spider-v1] INFO: Parse Item function called on https://www.42maru.ai/en/contact/ 2 2
2026-08-18 00:41:41 [root] INFO: 신규 페이지 저장
2026-08-18 00:41:41 [sb-spider-v1] INFO: Spider 'sb-spider-v1' closed. Reason : closespider_itemcount
2026-08-18 00:41:41 [sb-spider-v1] INFO: spider_closed function called
2026-08-18 00:41:41 [sb-spider-v1] INFO: reason - closespider_itemcount
2026-08-18 00:41:41 [sb-spider-v1] INFO: scraped_item_count - 3
2026-08-18 00:41:41 [sb-spider-v1] INFO: processed_item_count - 3
2026-08-18 00:41:41 [sb-spider-v1] INFO: no callback
2026-08-18 00:41:41 [sb-spider-v1] INFO: Message sent successfully to Slack!
2026-08-18 00:41:46 [sb-spider-v1] INFO: Update Not Crawled Page Status - D
2026-08-18 00:41:46 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 730,
 'downloader/request_count': 3,
 'downloader/request_method_count/GET': 3,
 'downloader/response_bytes': 154390,
 'downloader/response_count': 3,
 'downloader/response_status_count/200': 3,
 'dupefilter/filtered': 1,
 'elapsed_time_seconds': 51.979464,
 'finish_reason': 'closespider_itemcount',
 'finish_time': datetime.datetime(2026, 8, 18, 0, 41, 41, 622599),
 'item_scraped_count': 3,
 'log_count/INFO': 32,
 'memusage/max': 123355136,
 'memusage/startup': 123355136,
 'pipeline/items_processed': 3,
 'playwright/context_count': 1,
 'playwright/context_count/max_concurrent': 1,
 'playwright/context_count/non_persistent': 1,
 'playwright/page_count': 3,
 'playwright/page_count/max_concurrent': 1,
 'playwright/request_count': 191,
 'playwright/request_count/method/GET': 172,
 'playwright/request_count/method/POST': 19,
 'playwright/request_count/navigation': 4,
 'playwright/request_count/resource_type/document': 4,
 'playwright/request_count/resource_type/fetch': 4,
 'playwright/request_count/resource_type/image': 67,
 'playwright/request_count/resource_type/ping': 12,
 'playwright/request_count/resource_type/script': 77,
 'playwright/request_count/resource_type/stylesheet': 15,
 'playwright/request_count/resource_type/xhr': 12,
 'playwright/response_count': 190,
 'playwright/response_count/method/GET': 171,
 'playwright/response_count/method/POST': 19,
 'playwright/response_count/resource_type/document': 4,
 'playwright/response_count/resource_type/fetch': 4,
 'playwright/response_count/resource_type/image': 67,
 'playwright/response_count/resource_type/ping': 12,
 'playwright/response_count/resource_type/script': 77,
 'playwright/response_count/resource_type/stylesheet': 15,
 'playwright/response_count/resource_type/xhr': 11,
 'request_depth_max': 1,
 'response_received_count': 3,
 'scheduler/dequeued': 3,
 'scheduler/dequeued/memory': 3,
 'scheduler/enqueued': 32,
 'scheduler/enqueued/memory': 32,
 'start_time': datetime.datetime(2026, 8, 18, 0, 40, 49, 643135)}
2026-08-18 00:41:46 [scrapy.core.engine] INFO: Spider closed (closespider_itemcount)
2026-08-18 00:41:46 [scrapy-playwright] INFO: Closing download handler
2026-08-18 00:41:46 [scrapy-playwright] INFO: Closing download handler
2026-08-18 00:41:46 [scrapy-playwright] INFO: Closing browser