2024-03-25 03:20:38,267 INFO c.e.d.r.d.DatasetStore [post-executor-1131] Deleted batch with version number '268' from dataset with id 'a973102d-854b-4f54-9988-5266ba0919a6'.
2024-03-25 03:20:38,267 INFO c.e.d.r.d.h.DefaultDatasetBatchHousekeeper [post-executor-1131] Batch housekeeping result for 'source:dataset:a973102d-854b-4f54-9988-5266ba0919a6:37afbb1e-fe2b-4f8d-918d-ad968f92f2c8:db877c42-1e6f-4bc9-a118-2efa0104d6c0 v269' = {isKeepCurrentLoadingBatch=true, deleteVersions=[268]}.
2024-03-25 03:20:38,275 INFO c.e.d.w.c.LoadDataUtils [post-executor-1131] Load complete. Context: WorkflowId=204; WorkflowUuid=33a7600c-0bbe-41c3-aa1a-96549fa2903c; WorkflowName=wf_Deduplication_3_DatasetValidation; ExecutionId=7183a6fb-cd70-4e50-82a6-b0877bdd6bab; EnvironmentId=3; EnvironmentName=Dev; 
2024-03-25 03:20:38,299 INFO c.e.d.j.w.DefaultWorkflowProcessor [jobControllerSystem-pekko.actor.default-dispatcher-475] Executed workflow in 1381675 ms. Context: WorkflowId=204; WorkflowName=wf_Deduplication_3_DatasetValidation; ExecutionId=7183a6fb-cd70-4e50-82a6-b0877bdd6bab; EnvironmentId=3; EnvironmentName=Dev; 
2024-03-25 03:20:38,644 INFO c.e.d.j.w.DefaultWorkflowProcessor [jobControllerSystem-pekko.actor.default-dispatcher-478] Workflow execution starting. Context: WorkflowId=203; WorkflowName=wf_Deduplication_4a_HighMatchScoreDeduplication; ExecutionId=b66a6b00-b400-403a-acaf-6a01c3a6d448; EnvironmentId=3; EnvironmentName=Dev; 
2024-03-25 03:20:38,757 INFO c.e.d.r.i.f.FindDuplicatesEstablishResourceBuilder [workpool-server-fixmem-executor-worker-1170] Find duplicates establish operation started on store with id 'Dupstore_Deduplication_RemoteVM'.
2024-03-25 03:22:40,779 INFO c.e.d.r.i.f.FindDuplicatesEstablishResourceBuilder [workpool-server-fixmem-executor-closer-1170] Find duplicates establish operation on store with id 'Dupstore_Deduplication_RemoteVM' completed. Processing output...
2024-03-25 03:22:40,779 ERROR c.e.d.r.i.f.FindDuplicatesEstablishResourceBuilder [workpool-server-fixmem-executor-closer-1170] Find Duplicates step failed. org.apache.pekko.pattern.AskTimeoutException: Ask timed out on [Actor[pekko://match/user/root#2034671105]] after [120000 ms]. Message of type [com.experian.match.actorsys.messages.RunTearDown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.
2024-03-25 03:22:40,835 ERROR c.e.d.c.e.ErrorLogger [workpool-server-fixmem-executor-closer-1170] Error 24: 4009: Error closing builder
com.experian.datastudio.common.exceptions.ExecutionException: 4009: Error closing builder
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.closeBuilder(ExecutorServiceBuilderWorkPool.java:420) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.lambda$new$2(ExecutorServiceBuilderWorkPool.java:108) ~[resource-2.13.6.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:840) ~[?:?]
Caused by: java.io.IOException: Find Duplicates step failed. Please check logging output for more information
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.checkStatus(FindDuplicatesEstablishResourceBuilder.java:138) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.processResults(FindDuplicatesEstablishResourceBuilder.java:54) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesResourceBuilder.closeImpl(FindDuplicatesResourceBuilder.java:96) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.AbstractResourceBuilder.close(AbstractResourceBuilder.java:151) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.closeBuilder(ExecutorServiceBuilderWorkPool.java:416) ~[resource-2.13.6.jar:?]
	... 6 more
2024-03-25 03:22:40,902 INFO c.e.d.r.i.f.FindDuplicatesEstablishResourceBuilder [workpool-server-fixmem-executor-worker-1171] Find duplicates establish operation started on store with id 'Dupstore_Deduplication_RemoteVM'.
2024-03-25 03:24:40,870 ERROR c.e.d.m.MatchApiImpl [workpool-server-fixmem-executor-closer-1171] Error attempting to delete match store: 
com.experian.match.api.client.ApiException: {"matchStoreId":"Dupstore_Deduplication_RemoteVM","state":"FAILED","progress":100.0,"message":"Find Duplicates step failed. java.util.concurrent.TimeoutException: Future timed out after [120 seconds]","createTime":"2024-03-24T03:21:02.707596400","startTime":"2024-03-24T03:23:15.655582200","finishTime":"2024-03-25T03:24:40.865291100","matchStoreLocation":"E:\\ApertureDataStudio\\data\\experianmatch"}
	at com.experian.match.api.client.MatchResponseHandler.throwIfStatusDoesNotMatch(MatchResponseHandler.java:73) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.match.api.client.MatchResponseHandler.throwIfStatusNotOk(MatchResponseHandler.java:68) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.match.api.client.MatchResponseHandler.handleResponse(MatchResponseHandler.java:20) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.match.api.client.MatchApi.deleteMatchStore(MatchApi.java:206) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.datastudio.match.MatchApiImpl.deleteMatchStore(MatchApiImpl.java:104) ~[match-2.13.6.jar:?]
	at com.experian.datastudio.match.MatchService.deleteMatchStore(MatchService.java:224) ~[match-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.cancelEstablishmentJob(FindDuplicatesEstablishResourceBuilder.java:93) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.abortJobs(FindDuplicatesEstablishResourceBuilder.java:79) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesResourceBuilder.abortImpl(FindDuplicatesResourceBuilder.java:113) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.AbstractResourceBuilder.abort(AbstractResourceBuilder.java:123) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.stopBuilder(ExecutorServiceBuilderWorkPool.java:433) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.lambda$new$4(ExecutorServiceBuilderWorkPool.java:113) ~[resource-2.13.6.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:840) ~[?:?]
2024-03-25 03:24:40,981 INFO c.e.d.r.i.f.FindDuplicatesEstablishResourceBuilder [workpool-server-fixmem-executor-closer-1172] Find duplicates establish operation on store with id 'Dupstore_Deduplication_RemoteVM' completed. Processing output...
2024-03-25 03:24:40,981 ERROR c.e.d.r.i.f.FindDuplicatesEstablishResourceBuilder [workpool-server-fixmem-executor-closer-1172] Find Duplicates step failed. java.util.concurrent.TimeoutException: Future timed out after [120 seconds]
2024-03-25 03:24:41,038 ERROR c.e.d.c.e.ErrorLogger [workpool-server-fixmem-executor-closer-1172] Error 25: 4009: Error closing builder
com.experian.datastudio.common.exceptions.ExecutionException: 4009: Error closing builder
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.closeBuilder(ExecutorServiceBuilderWorkPool.java:420) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.lambda$new$2(ExecutorServiceBuilderWorkPool.java:108) ~[resource-2.13.6.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:840) ~[?:?]
Caused by: java.io.IOException: Find Duplicates step failed. Please check logging output for more information
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.checkStatus(FindDuplicatesEstablishResourceBuilder.java:138) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.processResults(FindDuplicatesEstablishResourceBuilder.java:54) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesResourceBuilder.closeImpl(FindDuplicatesResourceBuilder.java:96) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.AbstractResourceBuilder.close(AbstractResourceBuilder.java:151) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.closeBuilder(ExecutorServiceBuilderWorkPool.java:416) ~[resource-2.13.6.jar:?]
	... 6 more
2024-03-25 03:24:41,042 INFO c.e.d.j.w.DefaultWorkflowProcessor [jobControllerSystem-pekko.actor.default-dispatcher-478] Executed workflow in 242325 ms. Context: WorkflowId=203; WorkflowName=wf_Deduplication_4a_HighMatchScoreDeduplication; ExecutionId=b66a6b00-b400-403a-acaf-6a01c3a6d448; EnvironmentId=3; EnvironmentName=Dev; 
2024-03-25 03:24:41,067 ERROR c.e.d.j.a.WorkflowExecutionMonitorActor [jobControllerSystem-pekko.actor.default-dispatcher-482] Catching
com.experian.datastudio.jobcontroller.exception.JobControllerException: 9003: An unexpected error has occurred during Workflow execution
	at com.experian.datastudio.jobcontroller.actors.WorkflowExecutionActor.getWorkflowException(WorkflowExecutionActor.java:273) ~[jobcontroller-2.13.6.jar:?]
	at com.experian.datastudio.jobcontroller.actors.WorkflowExecutionActor.onExecuteWorkflow(WorkflowExecutionActor.java:202) ~[jobcontroller-2.13.6.jar:?]
	at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[scala-library-2.13.11.jar:?]
	at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[scala-library-2.13.11.jar:?]
	at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) ~[scala-library-2.13.11.jar:?]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ~[scala-library-2.13.11.jar:?]
	at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) ~[pekko-actor_2.13-1.0.2.jar:1.0.2]
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?]
	at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[?:?]
	at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[?:?]
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[?:?]
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[?:?]
2024-03-25 03:24:41,265 INFO c.e.d.j.w.DefaultWorkflowProcessor [jobControllerSystem-pekko.actor.default-dispatcher-482] Workflow execution starting. Context: WorkflowId=197; WorkflowName=wf_Common_Log_Error; ExecutionId=53e606e8-3a9f-485c-9ec2-373e4ceefcb6; EnvironmentId=3; EnvironmentName=Dev; 
2024-03-25 03:24:41,296 INFO c.e.d.w.e.o.SourceOutput [output-executor-pool-1] Auto-refresh for dataset 'ds_Common_Onside_Dedupe_Log' will be attempted
2024-03-25 03:24:41,515 INFO c.e.d.w.c.LoadDataUtils [output-executor-pool-1] Loading dataset. Context: DatasetUuid=d1af15d3-ad2b-4088-9db1-c52d7408eeb0; DatasetName=ds_Common_Onside_Dedupe_Log; EnvironmentId=3; EnvironmentName=Dev; LoadId=91b57d0b-0dff-4347-b6a6-ee84b36644da; 
2024-03-25 03:24:41,696 INFO c.e.d.r.d.h.DefaultDatasetBatchHousekeeper [workpool-server-lowmem-executor-closer-658] Batch housekeeping result for 'source:dataset:d1af15d3-ad2b-4088-9db1-c52d7408eeb0:e8041cf7-d93b-4d36-9be5-5294ae91ddb3:a34cc75e-e3e8-4064-aa6e-7e15838d4e8a v561' = {isKeepCurrentLoadingBatch=true, deleteVersions=[560]}.
2024-03-25 03:24:41,844 INFO c.e.d.r.d.h.DefaultDatasetBatchHousekeeper [LocalResourceManager-change-handler 1017525834] Batch housekeeping result for 'source:dataset:d1af15d3-ad2b-4088-9db1-c52d7408eeb0:e8041cf7-d93b-4d36-9be5-5294ae91ddb3:a34cc75e-e3e8-4064-aa6e-7e15838d4e8a v561' = {isKeepCurrentLoadingBatch=true, deleteVersions=[]}.
2024-03-25 03:24:41,852 INFO c.e.d.w.c.LoadDataUtils [LocalResourceManager-change-handler 1017525834] Load complete. Context: EnvironmentId=3; EnvironmentName=Dev; 
2024-03-25 03:24:42,172 INFO c.e.d.w.e.o.i.JDBCExportIOIterator [workpool-server-fixmem-executor-closer-1174] Applying post-SQL
2024-03-25 03:24:42,200 INFO c.e.d.w.e.o.i.JDBCExportIOIterator [workpool-server-fixmem-executor-closer-1174] EXPORT COMPLETED IN 78ms
2024-03-25 03:24:42,200 INFO c.e.d.w.e.o.i.JDBCExportIOIterator [workpool-server-fixmem-executor-closer-1174] BATCH SIZE: 1000
2024-03-25 03:24:42,200 INFO c.e.d.w.e.o.i.JDBCExportIOIterator [workpool-server-fixmem-executor-closer-1174] INSERT: 0
2024-03-25 03:24:42,201 INFO c.e.d.w.e.o.i.JDBCExportIOIterator [workpool-server-fixmem-executor-closer-1174] UPDATE: 1
2024-03-25 03:24:42,201 INFO c.e.d.w.e.o.i.JDBCExportIOIterator [workpool-server-fixmem-executor-closer-1174] DELETE: 0
2024-03-25 03:24:42,202 INFO c.e.d.j.w.DefaultWorkflowProcessor [jobControllerSystem-pekko.actor.default-dispatcher-482] Executed workflow in 906 ms. Context: WorkflowId=197; WorkflowName=wf_Common_Log_Error; ExecutionId=53e606e8-3a9f-485c-9ec2-373e4ceefcb6; EnvironmentId=3; EnvironmentName=Dev; 
2024-03-25 03:26:41,073 ERROR c.e.d.m.MatchApiImpl [workpool-server-fixmem-executor-closer-1173] Error attempting to delete match store: 
com.experian.match.api.client.ApiException: {"matchStoreId":"Dupstore_Deduplication_RemoteVM","state":"FAILED","progress":100.0,"message":"Find Duplicates step failed. java.util.concurrent.TimeoutException: Future timed out after [120 seconds]","createTime":"2024-03-24T03:21:02.707596400","startTime":"2024-03-24T03:23:15.655582200","finishTime":"2024-03-25T03:26:41.070509100","matchStoreLocation":"E:\\ApertureDataStudio\\data\\experianmatch"}
	at com.experian.match.api.client.MatchResponseHandler.throwIfStatusDoesNotMatch(MatchResponseHandler.java:73) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.match.api.client.MatchResponseHandler.throwIfStatusNotOk(MatchResponseHandler.java:68) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.match.api.client.MatchResponseHandler.handleResponse(MatchResponseHandler.java:20) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.match.api.client.MatchApi.deleteMatchStore(MatchApi.java:206) ~[match-api-all-3.10.2.001.jar:3.10.2.001-457091]
	at com.experian.datastudio.match.MatchApiImpl.deleteMatchStore(MatchApiImpl.java:104) ~[match-2.13.6.jar:?]
	at com.experian.datastudio.match.MatchService.deleteMatchStore(MatchService.java:224) ~[match-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.cancelEstablishmentJob(FindDuplicatesEstablishResourceBuilder.java:93) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesEstablishResourceBuilder.abortJobs(FindDuplicatesEstablishResourceBuilder.java:79) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.indexer.findduplicates.FindDuplicatesResourceBuilder.abortImpl(FindDuplicatesResourceBuilder.java:113) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.AbstractResourceBuilder.abort(AbstractResourceBuilder.java:123) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.stopBuilder(ExecutorServiceBuilderWorkPool.java:433) ~[resource-2.13.6.jar:?]
	at com.experian.datastudio.resource.ExecutorServiceBuilderWorkPool.lambda$new$4(ExecutorServiceBuilderWorkPool.java:113) ~[resource-2.13.6.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:840) ~[?:?]